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(57) Abstract 

The invention permits the generation of multipurpose control 
signals by tracking movement that occurs within a field of view. In 
particular, a "Guest Controlled Orchestra" utilizing these inventive 
principles permits a layman guest to step into the shoes of an orchest- 
ra conductor, and through image processing, conduct the performance 
of a prerecorded music score. A video camera captures a field of view 
encompassing the guest for generation of a digital image. The field of 
view is sampled in left and right windows and the intensity of pixels 
within the windows are compared with a past image to determine if in- 
tensity change exceeds a predetermined threshold. A center of move- 
ment is computed for each window by averaging coordinates of each 
such pixel, and the centers of movement stored for future use. By anal- 
yzing change in centers of movement, tempo and volume are derived. 
Volume is derived from the quantity of pixels that correspond to the 
predetermined intensity change, and which therefore represent move- 
ment. Prerecorded audio data are formatted into MIDI audio com- 
mands, and together with video frame advance commands, are pro- 
cessed and output in response to these derived signals. 
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APPARATUS AND METHOD FOR TRACKING MOVEMENT 
TO GENERATE A CONT ROL SIGNAL 

A portion of the disclosure of this patent document 
contains material which is subject to copyright 
5 protection. The copyright owner has no objection to the 

facsimile reproduction by anyone of the patent document or 
the patent disclosure, as it appears in the Patent and 
Trademark Office patent file or records, but otherwise 
reserves all copyright rights whatsoever. 

10 BACKGROUND 

The present invention relates to an apparatus and 
method for generating a control signal in response to 
movement, and more particularly, relates to a music 
generator that extracts information for "conducting" 
15 music, in the same sense that a conductor would conduct an 

orchestra . 

People enjoy music. Many especially enjoy classical 
music and appreciate the role of the conductor. 
Typically, a conductor will orchestrate music individually 
20 played by over one hundred independent instruments into 

one united harmonious score. The styles and products of 
different conductors are as unique as the music scores 
that they conduct. 

The conductor is viewed as the head of the 
25 orchestra, its leader, and frequently, is the recipient of 

praise for his creativity and his ability to transform the 
work of a composer into a derived, artistically unique 
musical product. 

Many have fantasized being a conductor and being 
30 able to create such a unique musical creation. However, 

for most, this fantasy will not be achieved, because of 
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the difficulty in learning and in mastering the 
conductor's unique "language". 

The" conductor must be fluent in expressing 
conducting information, used to produce the final musical 

5 product, to each member of the orchestra. This 

information is transmitted through the gestures and 
movements of the conductor and is based upon his knowledge 
and experience in music in general, his knowledge of the 
music score which is to be performed, his own style and 

0 taste, and the knowledge and experience and style of each 

member of the orchestra. Learning the conductor's musical 
experience and the other information necessary to conduct 
an orchestra is fairly difficult on its own, but in 
addition, the layman must also learn the language by which 

5 a conductor communicates his expression and taste to 

enable the members of the orchestra to play in unison at 
the correct tempo, volume, emphasis and presence. 

Thus, the conductor's musical skills necessary to 
synchronize and direct the playing of music typically 

0 exceed those of the common person. However, this 

inability does not eliminate the desire that many have to 
express their own musical taste and style. To this end, 
equipment and methods have been developed over recent 
years which attempt to enable a common person to step into 

:5 the shoes of the conductor, and to create a unique musical 

product, stylized by their own expression. 

One such system, employed for many years at "EPCOT 
Center," at Walt Disney World, Florida, enables one to 
simulate the role of conductor by mixing instrument tracks 
10 that correspond to a prerecorded musical score. Using 

this system, one assumes a defined spot and moves and 
gestures as if he or she were the conductor. Four sonar 
devices are used to each derive the relative distance of 
a hand intersecting a sonar beam to a background object 
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normally struck by the beam. This relative distance is 
then used to raise or lower corresponding tracks of a 
prerecorded musical score. 

Another method utilizes video processing to isolate 
5 the outline of the form of a guest conductor against a 

distinguished background and to derive information from 
the orientation of the person's outline. According to 
this method, the repeatedly captured outline can be 
analyzed for movement, and direction or speed of movement 
10 determined from recognized states can be extracted and 

used or analyzed to modify sound. 

Still another system attaches motion sensors to a 
guest conductor. These sensors, which may include, for 
example, motion sensing gloves or the like, sense 
15 acceleration or movement relative to other sensors, and 

thereby provide an electronic signal that can be used to 
generate music. 



Each of these systems has disadvantages that offer 
room for improvement and modification. For example, in 
20 the video system mentioned above, the guest conductor and 

a background must be specifically contrasted in order to 
allow the video equipment to provide a stark contrast to 
capture the guest conductor's outline. 

Other difficulties are also present in a video 
25 device. Video systems generally use image processing 

equipment that transforms the video signal into "pixels", 
i.e., numbers that each represent sampled visual 
characteristics at distinct locations scanned by a video 
camera. These video systems produce many thousands of 
30 "pixels" per second. Typically, the known methods of 

processing these pixels, such as by tracing only the 
outline, must rely on some shortcut in order to track 
movement. The huge number of pixels and complex 
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processing that is required make full image processing a 
practical impossibility for real time applications. 

With respect to the sonar system mentioned above, 
its field of view is extremely limited since the sonar 
5 waves are either directed to, or are received from, a 

specific point in order to accurately locate the guest 
conductor's hands and to distinguish them against other 
sound reflecting backgrounds. Furthermore, the nature of 
the sonar system dictates that only a limited range of 
10 guest activities will produce results. This limitation in 

the range of activities detracts from the guest's freedom 
of expression and makes it more difficult for a guest to 
create music by imitating his or her mental impression of 
a conductor in action. 

15 Accordingly, there has existed a definite need for 

an apparatus or method which can generate a control signal 
in response to movement and use that signal to alter the 
performance parameters of prerecorded music. Such a 
system would need to track movements which occur within a 

20 field of view. Additionally, it should provide a method 

for processing that employs a reference back to specific, 
previously identified pixels derived from a video image to 
enable a digital processing system to process the movement 
in real-time. This needed apparatus or method should be 

25 applicable in particular to a device that permits the 

controlled performance of music in response to tracked 
conductor parameters. 

The present invention fulfills these needs, 
overcomes many of the aforementioned disadvantages and 
30 provides an improved and unique apparatus and method for 

playing music and for allowing a guest conductor to direct 
a musical score. In broader terms, however, the present 
invention provides a unique and novel apparatus and method 
for generating control signals which may be applied to a 
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wide variety of tasks by tracking movements that occur 
within a field of view, 

SUMMARY OF THE INVENTION 

The present invention provides a novel control 
5 system that generates a control signal by tracking 

movement within a field of view, and which can track 
gestures and movements of a person occupying that field of 
. view. In this manner, movement may be utilized in a wide 
variety of applications to orchestrate system control 
10 through derived electronic signals. In particular, the 

specific contemplated application of the invention is to 
a Guest Controlled Orchestra that permits a guest to step 
into the shoes of an orchestra conductor and direct the 
performance of music that tracks his movements, much as a 
15 real orchestra would a professional conductor. 

The invention provides a digital motion processing 
system having an imaging device that repeatedly captures 
a field of view, which is then processed by image and data 
processing elements to yield relative movement between 

20 frames. From the comparison of the two images, the 

processing elements determine a single value 
representative of positions of the current image that 
correspond to movement. In response to this single value 
a signal generator generates at least one control signal. 

25 Since the preferred embodiment is a music system, the 

signal generator may be included in a sound generator that 
uses the generated control signals to generate and format 
sound . 

The method of processing information from a field of 
30 view to generate this control signal includes generating 

and storing a first image representing the field of view 
at a first point in time, and generating a second image 
representing the field of view at a second different point 
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in time. Pixels from these two images are compared to 
determine a set of pixels that represent movement of the 
second image relative to the first image. From the 
locations of those pixels in the set that represent 
5 movement, a single value is computed. The control signal 

is generated in response to this single value. 

More particularly, the apparatus utilizes an imaging 
device to provide an electronic signal representative of 
a field of view captured by the imaging system. This 

10 electronic signal, representing a sequence of scans of the 

field of view, is digested by an image digitizer that 
processes the electronic signal to yield a sequence of 
numbers. Each number corresponds to the intensity of a 
particular point in the field of view and thus represents 

15 a specific, addressable location, or "pixel", within that 

field of view. This numeric data is then processed in a 
series of steps to efficiently permit the tracking of 
movement within the field of view. 

The intensity of pixels of an image are compared 
20 with the intensity of a previous image at the same pixel 

location to determine those pixels that represent movement 
relative to the previous image. A difference in 
corresponding pixel values that exceeds a selected 
threshold is taken as an indication of motion. The data 
25 processor computes a single value representative of all 

the pixels that represent movement for a given image. 

At a more specif ic level of the invention, the data 
processor computes a single position, or "centroid", for 
each "window" or predefined subportion of the field of 
3 0 view which is to be specifically analyzed for movement. 

It then computes, from all such single positions, the 
single value, or "scalar", which it then analyzes to 
generate the control signal. For example, in the case of 
the preferred embodiment described below, x and y indices 
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ofi two such centroids for each image are simply summed to 
yield a single number, or "scalar", representing movement 
within the field of view. 



Following a more particular form of the invention, 
5 after processing several image frames, the relative 

magnitude of several of the single values representing 
recent frames is correlated over a larger group of the 
single values to determine how fast or slow prerecorded 
music should be played. By recognizing specific guest 
10 actions, more particular forms of the invention also 

provide for selection of different instruments which can 
be alternatively used to play the same notes, and for 
volume control. The latter is dependent upon the number 
of pixels for a given image frame that represent movement. 

15 Another feature of the motion processing system in 

accordance with the invention provides for control of a 
video display in response to tracked movement. For 
example, a prerecorded video image, such as a cartoon, may 
be visually displayed to the user or others at a rate that 

20 tracks the user' s movements occurring within the field of 

view. This latter embodiment correctly emphasizes the 
current invention as providing a method of generation 
control signals in general, with many possible 
applications . 

25 The invention may be better understood by referring 

to the following detailed description, which should be 
read in conjunction with the accompanying drawings. The 
detailed description of a particular preferred embodiment, 
set out below to enable one to build and use an example of 

3 0 the invention, are not intended to limit the claims but to 

serve as a particular example thereof. 
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DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a perspective view of an electronic guest 
controlled orchestra system in accordance with the 
invention; 

5 FIG. 2 is a perspective view of the system 

illustrated as in FIG. 1, with the components of a console 
shown in greater detail; 

FIG. 3 is a flow chart illustrating operation the 
main processing functions of a data processor used in the 
10 guest conducted orchestra system shown in FIG. 1; 

FIG. 4 is a flow chart' of the pixel sampling 
function, referred to in FIG. 3, that illustrates in 
greater detail the steps used to accomplish pixel sampling 
and cent ro id computation; 

15 FIG. 5 is a flow chart of a background schedule 

filling process for the MIB that operates in parallel with 
the main processing of the data processor and that 
controls the loading of MIDI note commands from a computer 
to the MIB used in the guest conducted orchestra system of 

20 FIG. 1; 

FIG. 6 is a flow chart which illustrates in greater 
detail a correlation process referred to in the flow chart 
shown in FIG. 3 ; 

FIG. 7 is a flow chart which illustrates maxima 
25 detection in accordance with FIG. 6; and, 

FIG. 8 is a flow chart which illustrates minima 
detection in accordance with FIG. 6. 
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DETAILED DESCRIPTION 



In accordance with the principles of the invention 
which have been summarized in the section above, the 
inventors have developed a preferred implementation of 
5 their invention which will be further described below. 

This preferred embodiment is called a "Guest Controlled 
Orchestra"/ and allows a guest to imitate the actions of 
a conductor, and to observe how those actions affect the 
generation of music and the .display of animation. The 

10 "orchestra" may be an ensemble of at least one orchestral 

voice, which may be an instrument or voice, or any other 
means of generating sound. Thus, the preferred embodiment 
may be applied to allow the guest to conduct nearly any 
ensemble, from a single instrument to a choir, to a rock 

15 group, etc. However, it is emphasized that the invention, 

as described above, relates not just to a music system, 
but is more broadly a system for generating control 
signals from movement. 

The preferred use of the Guest Controlled Orchestra 
20 is in a theme park, and for that reason, it has been 

designed with smooth and simple guest interaction and 
throughput in mind. Pursuant to these criteria, the 
system has been designed to provide intuitive operation to 
a guest, and so does not require an employee/operator or 
25 specific instruction before use. 



Referring first to FIGS. 1 and 2, the Guest 
Controlled Orchestra 10 includes a conductor station 12, 
on which a guest 14 may stand to perform conducting 
movements with his or her arms, and a console 15 spaced a 
30 predetermined distance D from the conducting station and 

facing toward the person. The console 15 conceals from 
the person a monitoring means for optically scanning 
movements of the guest and which is aligned towards the 
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conductor station for this purpose and a computer 
processing means for digesting the scanned movements of 
the guest and generating and applying control signals. 
The console 15 also conceals a music storage means for 
5 storing electronic digital information that represents a 

musical piece to be played and audio playing means for 
playing the musical piece in response to the electronic 
digital information, as modified by the computer 
processing means. 

10 More particularly, the system 10 includes a video 

camera 16, aligned to monitor the guest 14 through an 
infrared window 18 r a computer 20 for performing image and 
data processing tasks, a MIDI synthesizer 22, which 
converts MIDI format music information supplied by the 

15 computer into a signal for mixing by a mixer 24 and audio 

amplification by an amplifier 26, and a speaker system 28 
for ultimate rendition of a prerecorded musical score that 
has been varied in accordance with the guest's conducting 
actions. The computer 20 is fitted with an image 

20 digitizer add-on board for digitizing the camera's analog 

video output into a form that can be read and processed by 
the computer. In addition, the computer is also fitted 
with a musical/personal computer interface add-on board 
that controls the timing of music commands for the MIDI 

25 synthesizer. A video disk player 3 0 and monitor 32 are 

coupled to the computer to permit display of animation 
that is synchronized to the guest's tempo, as well as an 
additional monitor 34 that displays a special viewing 
image that represents detection of movement by the system 

30 10. 

The system 10 also includes a conductor statuette 36 
and appropriate suggestive materials, illustrated in FIG. 
1 as signs and notes 38, to indicate to the guest to 
imitate the actions of a conductor in order to achieve the 
35 desired result of generating music. It has been found 
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that such suggestive materials are necessary and 
sufficient to cause the guest to intuitively operate the 
Guest Controlled Orchestra. 

Within the conductor station 12, the guest occupies 
5 a field of view which is scanned by the video camera 16. 

The preferred embodiment captures this field of view and 
analyzes "windows" where independent movement is expected, 
such as one window about the left arm, and one window 
about the right arm. Through image processing, the 

10 guest's image is separately processed for each window so 

that movement of each arm yields control information used 
to process instrument, volume and tempo inf ormation that 
will influence the rendition of the prerecorded music 
score. The field of view may optionally be divided into 

15 as many windows as necessary that will yield distinct 

physical movement, such as movement of a finger, for 
example. These "windows" may be dynamic, or made to 
change in position relative to the field of view, to track 
objects and movement. In the Guest Controlled Orchestra, 

20 however, only two static windows are used to separately 

process movement of the left and right arms. 

Each window may be processed to yield one or more 
control signals. Alternatively, a single control signal 
may be the product of several independent windows. The 
25 Guest Controlled Orchestra yields four control signals 

which influence the playing of the musical score. These 
signals include tempo, left only volume, right only volume 
and volume for instruments to be applied to both speakers. 

With reference to FIG. 2, the contents of the 
30 console 15 are shown as removed from their normal position 

of concealment from the guest. The video camera 16 
repeatedly scans the field of view and thereby captures 
the guest's image through the infrared window 18. The 
computer 20 is coupled to the video camera 16, so as to 
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receive its thirty frame -per -second input and process it 
to generate MIDI format music commands, as well as 
commands to control the playing of a video disk- The 
console 15 also conceals the professional style video disk 
5 player 30, which receives these commands, i.e., 1-5 frame 

advances, and directs the display of prerecorded animation 
stored on the video disk that also is sequenced to the 
guest's actions. The computer 20 processes audio data, 
which is stored internally in its RAM, to format MIDI 

10 commands and set volume and tempo information for the 

ultimate rendition by the sound system. The synthesizer 
22 receives the MIDI output of the computer and generates 
an analog electronic signal which may be used to drive the 
speakers 28. Selection of the acoustic elements, namely, 

15 the MIDI synthesizer 22, mixer 24, audio amplif ier 26 and 

speakers 28, is well within the skill of one familiar with 
electronic music equipment and hence, will not be 
discussed in detail. 

The console 15 conceals the video disk player 30, 

20 which is a professional type video disk player that may be 

used both to record and play, and which is controlled by 
a control signal output by the computer. Video frames are 
output by the video disk player in response to the tempo 
information generated by the computer 20 and coupled to 

25 the disk player 30 via an RS-232 connector. The disk 

player's output feeds the video monitor 32 to display 
visual data stored on the disk player to the guest 
conductor 12 and observers . In the case of the preferred 
embodiment, the video monitor displays animation, 

30 including animated sea creatures, that appear to dance or 

otherwise perform in synchronization with the music and 
the guest conductor's actions. Derived control of video 
display is yet another example of the application of the 
invention to a system for generating control signals in 

35 general. 
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The processing hardware, including the image 
processor, the data processor and their supporting memory, 
is embodied in the personal computer 20, as modified with 
the addition of add-on boards, including an image 
5 digitizer and a MIDl/personal computer interface. More 

specifically, the video frame store chosen for the system 
10 is an "IVG-128 11 board which is available from Datacube, 
Inc., of Peabody, Massachusetts. The MIDI/personal 
computer interface is analogous to a "MPU4 01" board, sold 

10 under the designation "MQX-32M", available from Music 

Quest, Inc., of Piano, Texas, and will hereafter be 
referred to as the musical interface board ("MIB"). The 
personal computer used in the preferred embodiment is an 
IBM 386-25 personal computer. Graphics and data 

15 processing software, written by the inventors in the "C" 

language and partly in machine language, directs the CPU's 
performance of the various tasks and the CPU' s interface 
with these two add-on boards. The software, which is 
illustrated in the flow charts shown in FIGS. 3-8, is 

20 described in functional terms in the paragraphs that 

follow. 



The video camera 16 chosen is a model "TI-24A" , 
available from the NEC Corporation of Tokyo, Japan. It 
generates a black and white electronic output of standard 

25 video format, elaborated upon below. While the preferred 

embodiment uses a black and white camera, which will 
typically capture portions of the infrared spectrum, any 
type of camera may be used in accordance with the 
principles of the invention. The infrared window 18, 

30 discussed above, is implemented not for the desirability 

of capturing the infrared spectrum, but primarily to hide 
the video camera from observance by the guest and yet 
allow the camera to capture the field of view. 



35 



The video camera 16 generates a standard video 
signal by scanning the field of view and producing sixty 
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interlaced fields per second, or thirty complete frames 
per second. An electronic signal is produced by the video 
camera which represents the monochromatic luminance of the 
field of view as it is line-scanned from left to right. 
5 Each scan is slightly offset vertically from other scans, 

such that five hundred and twenty- five horizontal lines of 
scanning are used to cover the entire field of view. The 
electronic video signal/ which repeats itself for each 
frame or thirty times per second, represents a generally 

10 continuous trace of the picture of five hundred and 

twenty-five horizontal lines when placed adjacent to one 
another. More precisely, each frame is comprised of two 
interlaced fields of alternating lines: The camera scans 
twice for each frame, scanning every other line each time, 

15 and produces five hundred and twenty- five lines. By 

scanning a picture in discrete lines comprising interlaced 
fields, the video camera produces a video electronic 
signal from a picture in approximately the reverse manner 
that a television reproduces a picture from a video 

20 signal. 

This video signal is coupled to the personal 
computer 20 for image and data processing. During receipt 
of the video signal from the video camera 16, the 
computer's ir IVG-128" board digitizes the video signal and 

25 thereby generates a plurality of pixels that represent the 

video signal. A "pixel" is nothing more than a sample of 
the video signal from which luminance can be discerned for 
a particular location within the field of view. Since the 
system 10 is a digital system, each video signal is 

30 digitized to provide a number for each pixel. This number 

comprises 8 bits that identify a sampled monochromatic 
(black-and-white) luminance for that pixel. 

Thus, the " IVG-128" board of the computer 20 
converts the repeated electronic video signal from the 
35 imaging system into a sequence of numbers, each number 
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corresponding to the luminance of the field of view at a 
particular point, or pixel. As will be discussed further 
below, each pixel corresponds to a location and is 
represented by coordinates that represent its location 
5 within the field of view. 



Thus, as described above, the video camera produces 
an electronic signal that scans left to right across the 
field of view producing approximately 262-1/2 lines of 
visual information, or a single interlaced field, sixty 
10 times per second. In addition, the electronic signal from 

the video camera contains synchronization information that 
is utilized by equipment receiving the electronic signal 
to reconstitute the visual image within the field of view. 

In digitizing the video signal, the "IVG-128" board 
15 produces two types of digital information in response to 

this electronic signal for processing by the computer. 
First, it produces 8 bit digital values that represent the 
sampled black- to-white luminance of a pixel, or of a 
particular position within the captured field of view. 
20 Second, it produces status flags, or bits for sampling by 

the computer, which indicate when the pixel generator has 
digitized each of the first and second interlaced fields 
of a given frame of visual data. It is necessary for the 
computer to monitor these status flags to wait for and 
25 synchronize the commencement of its main program loop with 

the digitizer's completion of the first interlaced field. 

The visual pixel information is stored in four 64 k 
banks of random access memory (which can be looked at as 
a 512 by 512 single byte frame buffer) , resident on the 
30 "IVG-128 11 board. Like any other type of random access 

memory ("RAM"), the pixel values may be overwritten with 
digital information to modify the stored image. The 
system 10 makes use of this ability in providing an output 
to the second monitor 34 . 
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The computer's GPU compares a defined subset, i.e., 
every fourth pixel every sixth line, of these pixels with 
corresponding information of a previous image that has 
been stored in the computer's random access memory* In 
other words, the computer compares the pixel's number 
representative of luminance with a number representing 
luminance at the same coordinates from a previous image. 

These sampling steps, described below, are described 
as including fixed increments. However, these constants 
are defined at the beginning .of the program, and may be 
chosen in the discretion of the computer operator to be 
any practical value. These constants are described below 
as specific values, because it is these values which have 
been used in operation of the preferred system. 

To sample and process the pixels, the computer first 
waits for the 11 IVG-128" to raise a status flag that 
indicates completion of the first of the two interlaced 
fields that make up each video frame. It is only this 
first field that is sampled by the computer's CPU for 
visual data. The CPU starts with column 25 (of 384 
columns that contain visual data) and row 50 (of 485 rows 
that contain visual data) and reads every fourth pixel 
until forty pixels have been read. It then moves six 
lines below, i.e., column 25, row 56, and repeats this 
same procedure. When fifty rows of forty horizontal 
samples are read, i.e., a left window of the image 
corresponding to the location of the right arm, the CPU 
reads another 40 x 50 sample block, beginning with column 
208, row 50, and proceeding every 4th pixel, every 6th 
row, to develop a right window of the image corresponding 
to the left arm. 

The 11 IVG-128" board stores the digitized image in an 
address format with the least significant address bits 
containing the horizontal coordinates, i.e., 0-lFF Hex, or 
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0-511, and the most significant address bits, i.e., 200- 
3FE00 Hex, or multiples of 512, containing the vertical 
coordinates, or row information. This information is 
organized into four banks of 64k random access memory and 
5 requires selection of a particular bank in accordance with 

the "IVG-128" ' s specifications and a standard sixteen bit 
address and CPU read and write operations. As indicated, 
although there are 512 possible columns or horizontal 
positions for each row, only 3 84 pixels contain visual 

10 information and only 80 of these are looked at by the CPU. 

Similarly, while the video signal has 525 interlaced lines 
per frame, only 485 contain visual information. It is 
therefore seen that by sampling only odd rows, the CPU 
scans only the first interlaced video field, and must 

15 complete its processing tasks before the completion of 

digitization of the next first interlaced field. 

As the number representing each pixel is read by the 
CPU from the n IVG-128 ll, s memory, it is compared with a 
number representing the same pixel of a previous frame, 

20 saved to the computer's random access memory. After 

performing the comparison steps, described below, the 
computer writes the new number over the .old number 
representing the previous frame, such that the new frame 
samples are stored in the computer's RAM and serve as the 

25 "previous frame" pixel data the next time the computer 

performs a comparison, one -thirtieth of a second later, 
for the next frame. It does this for each pixel that was 
used in the comparison regardless of magnitude of the 
luminance change, and thus requires 2 x 40 x 50 bytes of 

30 memory, or 4 k RAM, for the task. 

In this manner, the computer ascertains which of the 
sampled subset of pixels represent movement relative to 
the earlier image. "Movement" is represented by a change 
in intensity, or luminance, as referred to above. 
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Luminance will change somewhat for most pixels of 
the captured image. It is therefore necessary for the 
computer to identify only those pixels for which the 
change in luminance is sufficiently great that the new 
5 luminance represents actual movement, as opposed to a 

slight change in lighting, or the like. To this end, as 
indicated in the software block diagram of FIG. 4, the CPU 
subtracts the luminance of a particular pixel from the old 
number corresponding to that same pixel, i.e., the same 

10 location within the field of view, from the immediately 

previous image . If the absolute value of the difference 
exceeds a predetermined number, the computer adds special 
x-y indices corresponding to that pixel to a x index sum 
and a y index sum of all pixels that likewise are 

15 sufficiently different from the previous image for that 

window. In the Guest Controlled Orchestra, the +/- 
difference between numbers is compared with both 10 hex 
and F0 hex, rather than the above-mentioned use of the 
absolute value of the difference, as computation of the 

20 absolute value is equivalent but requires additional 

software steps. 

The computer writes each new pixel into the pixel 
sample buffer corresponding to each window, until all such 
pixels are exhausted. Therefore, in the Guest Controlled 
25 Orchestra, the computer 20 performs these steps for each 

sampled pixel of each of the left window and the right 
window. When it finishes each window, it will have a x 
index sum and a y index sum corresponding to that window. 

In addition, when the computer 20 determines that a 
30 pixel has changed its luminance sufficiently from the past 

image, it increments a pixel count corresponding to the 
analyzed window. In other words, the computer begins the 
pixel count at zero each time it begins analyzing a window 
for movement. It then keeps track of the number of pixels 
35 of each window that represent movement. Each of the x 
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index sum and y index sum corresponding to that window are 
subsequently divided by this "pixel count " . The result is 
a single x and y index value that represents a "centroid" , 
or single position within the window that is the average 
5 of all points which changed significantly in luminance, or 

which represent movement. As mentioned, centroids are 
computed for each window frame. As will be discussed 
further below, the system 10 uses the pixel counts for 
each window to also determine the audio volume 
10 instruments, or orchestral voices, of the music played. 

The x and y index corresponding to each sampled 
pixel should not be confused with its video row (of 512) 
and column (of 512) coordinates which necessitate an 18- 
bit address. Rather, the CPU performs a simpler task of 
15 assigning a row number, commencing with 49 and decreasing 

to 0, and a column number, commencing with 39 and 
decreasing to 0, which it uses for its processing tasks. 

As shown in FIG. 4 and the appended software 
listings, the CPU looks at the pixels in step wise 

20 fashion, as described above, in three stages. For each 

window, the CPU first searches for a valid pixel, 
comparing each new pixel sample with the corresponding old 
pixel sample until it detects a difference greater than 10 
hex or less than F0 hex, writing each new pixel into the 

25 pixel sample buffer as it does so. When the CPU has found 

a first valid pixel, it samples 10 rows each stepped 6 
lines apart, adding x and y index values of qualifying 
pixels to a corresponding cumulative sum and increasing 
the pixel count, as described above. Finally, after all 

30 ten rows have been sampled, the CPU ceases its comparison 

and testing functions and merely writes each remaining new 
pixel (in the 40 x 50 sample window) to the pixel sample 
buffer, overwriting the corresponding old pixel values as 
it does so. If at any time the CPU reaches the 2000th 

35 pixel for the window, i.e., row index = 0 and column index 
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= 0, it determines that it has reached the last sample , 
ceases all processing functions, and proceeds to the 
second window. As the CPU finishes each window, it 
computes a "centroid" , or movement center point for that 
5 window. Again, although the sampling for the second 

window commences at column 208, line 50 for the second 
window, the CPU defines an index value of y = 49, x = 39, 
and decrements these coordinates as the second window is 
sampled, to compute a second centroid corresponding to the 
10 second window. 

Thus, as the guest 14 waves each of his left and 
right arms, the computer 20 tracks these movements and 
generates a single point, thirty times per second for each 
of the left and right image windows , to represent movement 

15 relative to the immediately preceding frame. Ideal 

operation of the system makes some assumptions. The first 
is that the guest 14 remains at the conductor station 12 
so that his movement is captured by the video camera. 
Second, it is theoretically possible for the guest to 

20 swing his arms across his body in such a manner that the 

sum of both centroids produces a constant sum, in which 
case the system will detect a minimum tempo. It is 
stressed, however, that the system as described is robust 
against the guest's arms crossing his body so that any 

25 motion, including body swaying, will enable detection of 

a tempo. 

In the Guest Controlled Orchestra, the field of 
view, as digitized, is sampled in static left and right 
half windows, which capture the guest's movements as long 

30 as he remains in the conductor station 12. Thus, the same 

subset of pixels for each window are always used, and 
their values subsequently stored in computer' s RAM, for 
use as the prior image in the subsequent comparison step 
for the next frame. Alternatively, the windows analyzed 

35 could be small areas of the field of view and could be 
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made to be dynamic, or centered around prior movement, and 
thus made to track independent movements within the field 
of view. In this latter case, the computer would need to 
preview each dynamic window within the field of view and 
5 anticipate and store into memory those pixel values which 

it will need for the subsequent comparison, which may not 
be coextensive with those pixels used for other 
comparisons. As shown by the attached software appendix 
G, fixed pixel sample step sizes are defined at the outset 

10 of the subroutine and associated with variables. However, 

it would be well within the skill of one familiar with 
writing computer software to implement a subroutine which 
would adjust the value of the pixel step size variables 
during the program operation. Implementation of this or 

15 other alternate embodiments which make use of dynamic 

windows would be well within the ordinary level of skill 
in computer science or electronics. 

In addition to overwriting the 2000 samples for each 
window one-by-one into the computer's RAM, the computer 

20 also provides the above-mentioned second monitor output. 

In the preferred embodiment, the computer is also coupled 
to a second display monitor 34 that permits observers to 
observe the operation of the Guest Controlled Orchestra. 
After the computer has compared the difference between 

25 pixel values with 10 hex and F0 hex, the computer also 

writes a predefined luminance value into the "IVG-128 l "s 
memory associated with the sampled pixel. For example, if 
the result of the comparison is that the difference is 
either less than 10 hex or greater F0 hex, a grey value 

30 (80 hex) is written into the sampled pixel location. If 

the pixel change is greater than 10 hex or less than F0 
hex, indicating movement, a white value (FF hex) is 
written into the sampled pixel location. 



35 



The "IVG-128" board is configured to independently 
provide an analog video output that may be used to 
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direcfcly drive a video monitor. Thus, the second video 
monitor 34 is coupled to this output and requires no 
intervention by the computer's CPU to display information 
stored in the " IVG-128" ' s memory. As a result of its 
5 comparison step, however, the computer changes the sampled 

pixel value to either white or grey. Thus, on the display 
monitor, the concentration of white pixels will readily be 
observed superposed on the black- to -white image as 
indicating movement, and concentrations of grey pixels 
10 . observed as indicating regions where movement has not 

occurred. 

Once the computer's CPU has finished a 2000 sample 
window and has overwritten the last sample into the pixel 
sample buffer, the CPU will compute the centroid 

15 coordinates for that window. The CPU computes a centroid 

for each window by dividing each of the x index sum and 
the y index sum for each window by the window' s qualifying 
pixel count. It then stores this centroid coordinate in 
memory for subsequent use in computing a scalar, described 

20 further below. If no qualifying pixels have been found 

for the window, the CPU will utilize the centroid for the 
previous frame. 

Once both windows have been processed, the CPU 
satisfies itself that at least four qualifying pixels have 
25 been found over both windows. If less than four such 

pixels have been found, the CPU abandons its centroid and 
correlation steps and loads a minimum tempo to the MIB, as 
illustrated in the flow chart of FIG. 4, and then proceeds 
to process any video frame advance. 

30 After computing the centroid for each window, the 

CPU will determine a current volume for each channel used 
of eight possible channels, each corresponding to an 
orchestral voice. 
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Each MIDI command either turns an electronic note on 
or off. The "note on" commands, which are segregated into 
channels that each correspond to an orchestral voice, also 
contain digital information as to the volume of the note 
5 to be produced. The Guest Controlled Orchestra uses pixel 

counts to define volume of instruments appearing from each 
speaker and updates this information as it has finished 
looking at both windows. 

To update the volume corresponding to "note on" 
10 commands of each of the eight channels, the CPU looks at 

each window' s pixel count to determine the volume that is 
to be associated with certain instruments. For example, 
in the Guest Controlled Orchestra, the left channel is 
used for bass, and accordingly, the CPU will store a 
15 volume level derived from the number of qualifying pixels 

for the left window into a channel variable corresponding 
to bass. 

MIDI data is formatted to include data representing 
pitch (128 possible) , instrument type (16 possible) and a 

20 command that turns the note on or off. Thus, whenever the 

CPU is called upon to send a "note on" command, to the MIB, 
it simply replaces the bits representing volume with a set 
of bits derived from the pixel count, depending upon the 
type of instrument that the particular MIDI data 

25 represents. In current use of the Guest Controlled 

Orchestra, four instruments are used, including flute, 
bass, marimba, and drum. However, any type or number of 
instruments can be used. The flute is panned to the right 
channel and accordingly, the volume bits associated with 

30 MIDI flute commands will be derived entirely from the 

right window changed pixel count and put into a channel 
variable associated with the flute for use during the one- 
thirtieth second program cycle. Similarly, the volume of 
MIDI bass instrument commands will be substituted with a 

35 value derived from the changed pixel count in the left 



WO 93/22762 



PCT/US93/03667 



-24- 

window. The changed pixel counts for each of the left and 
right windows are averaged to yield volume information for 
marimba and drum note commands. Alternatively, rather 
than the preferred step of replacing the MIDI command bits 
5 associated with note volume, the computer's CPU could be 

instructed to amplify the command' s default volume by an 
amount dependent upon the changed pixel count. 

The system 10 uses two categories of interrupts, 
which request the CPU to cease performance of all program 
10 tasks and perform interim processing. The most 

significant of these CPU interruptions is utilized to 
direct filling the MIB's "schedule" of notes to be played. 
Operation and use of these interrupts may be understood 
with reference to FIG. 5. 

15 FIG. 5, rather than describing a subroutine or 

expanded processing step otherwise referred to in FIG. 3, 
illustrates the flow of the MIB that is distinct and 
collateral to the main program loop of FIG. 3 . When the 
system is initialized, the MIB has no commands in its 

20 buffers, or slots/ and the CPU will format and load MIDI 

commands into eight buffer pairs, defined in , RAM by the 
CPU. Each of these buffer pairs includes a "note on" 
buffer and a "note off" buffer and corresponds the eight 
scheduling slots of the MIB. Each of these eight 

25 "channels" will correspond to a particular instrument. 

When the CPU retrieves audio data from RAM and formats it 
to a MIDI command, the CPU detects the instrument type and 
places the command into either the "note on" or "note off" 
buffer corresponding to that instrument. Once the CPU has 

30 loaded at least twelve "note on" commands corresponding to 

each channel into its "note on" buffer (and corresponding 
"note off" commands into the corresponding "note off" 
buffer) , the CPU sends a command to the MIB directing the 
MIB to play music. 
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Operation of the MIB is the initiated by the CPU 
command, which triggers a generic interrupt from the MIB 
to tell the CPU that its slots are empty. The CPU feeds 
a command to the MIB for each channel used, along with a 
countdown time associated with the command. When the 
countdown time has expired and the command sent from the 
MIB to the synthesizer, the MIB generates a channel 
specific interrupt, which directs the CPU directly to the 
corresponding "note on" and "note off" channel buffer 
pair. The CPU looks at each of these buffers to ascertain 
which buffer holds the command to be issued the soonest. 
The CPU retrieves this command and formats and sends it to 
the empty MIB slot. Operation of the Guest Controlled 
Orchestra currently uses only four of these eight 
channels, assigned to flute, bass, marimba and drums. 

The "slots" described are a set of buffers resident 
on the MIB that each output a "note on" or "note off" 
command after a "countdown" time associated with the notes 
has elapsed. This countdown time is defined in "ticks", 
20 with 192 ticks per musical beat. The tempo given to the 

MIB determines the rate at which the MIB counts "ticks," 
and hence determines the rate at which notes are played or 
turned off. 

The "MQX-32M" used as the MIB is a standard musical 
25 interface board which receives MIDI commands formatted by 

the computer's CPU, and which puts those commands into one 
of eight scheduling buffers along with each command's 
associated countdown time. As stated, when the countdown 
time associated with the command has elapsed, the MIB 
30 removes the command from its associated buffer and feeds 

the command to the music synthesizer, triggering a channel 
specific interrupt to the CPU. The MIB thus interrupts 
the CPU at various intervals seeking a subsequent note 
command. Once note commands for a particular channel have 
35 expired, no new command is sent to the corresponding MIB 
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slot, which remains idle until reactivated by system 
initialization and a new command, prompted by the generic 
interrupt, described above. 

As the main program loop cycles through every one- 
5 thirtieth of a second, the CPU will access its pointer 

that points to the next digital information in RAM and 
look at that information stored in the score sequence. 
Each piece of digital information is stored as a nine byte 
command, including channel (or orchestral voice) , velocity 
10 (volume) , pitch, note start time and duration. The CPU 

analyzes channel type and ascertains by looking at the 
corresponding channel "note on" buffer whether that buffer 
has its allotment of twelve "note on" commands. If not, 
the CPU retrieves the nine byte word and increases a 
15 pointer to point at the next piece of digital information 

in the score sequence* 

The CPU must define "note on" and "note off" command 
times from each nine byte word and store respective 
commands in the channel's associated buffer pair. The 
constant chosen to define "note on" buffer length defines 
twelve slots that include an absolute time (four bytes) , 
a default volume, which may be altered as the "note on" 
command is output from the buffer, and pitch. The "note 
off" buffer is filled with a corresponding "note off" 
command, which must be properly inserted into a sequence 
of "note off" commands since note durations may vary. In 
other words, since successive note start times will be in 
sequence, but notes may vary in their duration, associated 
note end times may be out of sequence . The "note off" 
buffer for each channel is defined as twenty slots, which 
each store absolute command time and pitch. The absolute 
time for the "note off" commands is obtained^ by summing 
the absolute time of the corresponding "note on" command 
with the duration time. Each of these buffers is a 



20 



25 



30 



WO 93/22762 



PCT/US93/03667 



-27- 

circular buffer and has a pointer associated with it by 
the CPU. 



Each command is processed in this manner to f ill the 
CPU' s buffers with sufficient notes to last for the next 
5 one-thirtieth second cycle. 



When the CPU receives a channel specific interrupt 
from the MIB that tells the CPU to remove a note command 
from one of its corresponding channel buffer pair, the CPU 
first compares the next command for each of the "note on" 

10 and "note off" buffers corresponding to the channel, or 

orchestral voice . The command to be executed sooner is 
removed. If a "note on" command, the volume represented 
by one of the bytes of the command is over-written with the 
current computed volume that has been associated with that 

15 channel. Also, the countdown time is computed by the CPU 

by subtracting the absolute time of the channel's previous 
command, which is stored by the CPU, from the current 
absolute time of the "note on" or "note off" command. 
This countdown time is sent as part of the command to the 

20 MIB and the absolute time of the command stored by the CPU 

for further use. 



When the CPU is outputting a command to the MIB and 
computing the command's countdown time, the CPU compares 
the countdown time to 240 ticks. This is necessary, since 

25 the MIB can only hold a 240 tick countdown time. If the 

command's time is greater than 240 ticks, the CPU does not 
increment the note on/off pointer to the chosen command 
and instead sends an overflow command to the MIB. The CPU 
retains an absolute time of the overflow command for 

30 subsequent comparison with the next command. The overflow 

command is associated with a 240 tick countdown time which 
triggers no action by the synthesizer, but which causes 
another MIB interrupt corresponding to the same channel 
when the 240 ticks have elapsed. 
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A mentioned contemplated alternative embodiment also 
analyzes the guest's actions to select instrument , for 
example, by analyzing the vertical range of the guests 's 
movement . Thus , the vertical range or location of a 
5 guest's action could be made to influence the changing of 

notes having bass as the default instrument to horn notes, 
etc. Such a procedure could be implemented, for example, 
when the CPU removes a command from a channel buffer pair 
and formats the command for the MIB . This contemplated 
10 option has not been implemented in the current system, but 

it is within the principles of the invention. 

With the MIDI command appropriately formatted and 
stored for output to the MIB from the CPU's buffer, the 
MIB's interrupt is reset and the CPU returns to its normal 
15 program operation. 

The MIB operates on supplied tempo information and 
will employ the most recently provided tempo information 
to schedule note output . Tempo information is defined as 
a number of beats per minute, which may vary with the 
20 particular music to be played. In the preferred mode of 

the Guest Controlled Orchestra, the audio data represents 
a song with 240 beats per minute. 

MIDI format specified minimal temporal resolution is 
192 ticks per beat. Thus, in the preferred embodiment the 

25 MIB will schedule notes with each tick and will adjust its 

internal clock according to the tempo of the guest's 
actions in relation to 240 beats per minute. For example, 
if the guest moves his arms at a fast rate, the tempo will 
be perceived as being greater than 240 beats per minute, 

30 etc. The MIB will read a tempo word fed to it at the end 

of the main program cycle, every one- thirtieth second, and 
will perceive changes in the guest's rate of movement, 
adjusting its own internal clock accordingly. Thus, the 
MIB counts ticks faster or slower in waiting to output 
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channel commands to the synthesizer, depending upon the 
guest's rate of motion. 

To detect and ascertain this tempo information, the 
CPU must quantify movement occurring since the previous 
5 frame and must correlate this movement to ascertain a 

pattern of movement. To do this, the CPU looks at the 
mean location of perceived movement, which will tend to 
form a path over time. The CPU correlates the position of 
the current location of movement with the path over time, 
10 or history, to determine when it was last at a similar 

location^ Since each location is associated with a frame, 
or one -thirtieth of a second, the CPU can compute the rate 
of the guest's movement and a current tempo. 

When the CPU has computed the centroids for each of 

15 the left and right windows, it computes a scalar, or 

single value, which is a measure of the magnitude of all 
movement occurring within both windows. In order to 
capture all of guest's movement, which may be both 
vertical and horizontal, the CPU sums all coordinates for 

20 all of the centroids within the field of view. Thus, the 

CPU will add the y index of both the left and right window 
centroids, as well as the x index. However, since the 
guest will typically move the left and right arms together 
vertically, but in opposite directions horizontally, the 

25 x index for the centroid for the right window is inverted 

by taking its l's complement before summation. In this 
manner, opposite movements of the left and right arms in 
the horizontal direction will not cancel each other out, 
and the resultant scalar will be a measure of both 

3 0 vertical and horizontal movement of both arms. 



The CPU stores the scalar for each frame in a two 
hundred position circular buffer, defined in the 
computer's RAM. Thus, the computer's CPU will have a 
long-term memory of the most recent two hundred frames and 
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all older data will be overwritten by the most recent 
frame's scalar. 

When the program is initiated, the CPU's pointer 
which accesses the MIDI audio data stored in the 
5 computer's RAM is initialized to point at the first note 

of the song. Similarly, the various memory allocations 
corresponding to buffers to be used with data processing 
are zeroed. When the computer first detects a guest's 
presence and subsequent movement , it will load the MIB 

10 with a predefined tempo word read from its RAM and 

corresponding to the ideal rate of play of the music in 
the computer's RAM. During this time, the CPU analyzes 
the guest's movements to develop a history and to store 
this information as scalars in the circular buffer. Once 

15 two bars have been played, the CPU is free to change the 

tempo, as stored in the MIB, and thereby change the rate 
of play of the song. This "two bar" feature implemented 
as an inhibit which, after the computer has per formed the 
processing and correlation steps described above and 

20 below, will prevent the CPU' s update of the sixteen bit 

tempo word to the MIB. 

After the computer's CPU has written the newly 
processed scalar into the two hundred position circular 
buffer, it then correlates the most recent thirty scalars 

25 over the entire two hundred scalars in memory, as shown in 

FIG. 5. It does this by utilizing an index to point at 
the beginning sample of a thirty sample dynamic window 
over the two hundred scalars, and by performing the 
following process until one-hundred and seventy iterations 

30 have been performed and all two hundred samples 

correlated. The CPU: 

(1) subtracts 30 scalars beginning with, the scalar 
defined by the index from the most recent 30 
scalars , respectively; 
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(2) squares each of the 3 0 differences obtained and 
sums all 30 squares together; 

(3) stores the sum in a 170 position buffer; 

(4) increases the index to point to the next scalar 
5 for the next iteration; and 

(5) continues with 170 iterations until the index 
again points to the most recent sample; to save time 
each iteration subsequent to the first is computed 
by taking the square of the difference of the oldest 

10 scalar within the 3 0 sample window with the 3 0th 

most recent sample, adding that to the most recent 
sum, and subtracting the square of the difference of 
the most recent scalar with the newest scalar in the 
thirty sample window. 



15 This correlation process will thus produce one- 

hundred and seventy values which are each sums of the 
squares of the differences between two varying pair groups 
of thirty samples. Each of these two hundred numbers will 
vary from a value of zero, i.e., the two thirty sample 

20 windows are identical and thus when subtracted produce a 

value of zero, to a value which may be extremely high, and 
which is thus provided for by allocating 4 bytes of memory 
to each of the two hundred memory positions. Thus, the 
second buffer begins with the value of zero (both thirty 

25 sample windows coterminous) and will ascend to a very high 

number. When a scalar pattern similar to that represented 
by the most recent frame of data is detected, the sum of 
the squares of the differences will again be at a minimum. 
Thus, the positions are analyzed to obtain local minima 

30 corresponding to the guest's movements which were similar 

to that captured by the most recent series of video 
frames, and tempo extracted. 
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Since the first of the two hundred summed squares is 
always zero, the computer's CPU analyzes the increase in 
the squares by looking for a maximum and a subsequent 
minimum in two sequential program loops. When it detects 
5 a potential maximum , by determining that the next value is 

less than the preceding value , the CPU applies a bandgap 
to the maximum to make sure that it is followed by a 
downward trend differing by at least the number 512, It 
does this to guard against spurious maxima caused by 
10 noise. When it detects a potential minimum, it filters 

spurious minima by testing subsequent sums to ensure that 
the potential minimum is followed by an upward trend 
differing by at least the number 512. The CPU stops 
searching once it has obtained a second minimum. 

15 Two minima are sought in the preferred embodiment, 

because it is expected that a guest's arms will be moved 
both vertically and horizontally. As a result of this 
movement, the sums of the squares of the differences may 
experience either one or two minima associated with each 

20 cycle of typical motion by the guest. To correctly 

identify the guest's tempo, the CPU compares the first two 
minima to ascertain which is smallest, which it presumes 
to identify the correct tempo. This minimum is compared 
with a cut-off (selected in the preferred system as the 

25 number 2048) to ensure that the detected minimum is small 

enough to represent a reasonable correlation. If it is 
not, the correlation result is ignored and the tempo is 
not updated. 

With each correlation, the computer's CPU looks for 
30 the first two beats to generate the tempo information. 

Each squared sum corresponds to a location in the history 
buffer and is thus associated with time, a particular 
video frame and a particular point along the scalar' s 
periodic magnitude. The CPU, by performing the 

35 correlation process, effectively matches the scalar' s 
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change in magnitude at any particular point with a 
corresponding change of a different cycle, and thereby 
determines periodicity. The computer's CPU uses this 
information to compare the beat information derived from 
5 periodicity of the guest's movement with the ideal tempo 

for the particular musical piece. If the guest's tempo 
deviates from the most recent tempo information that the 
CPU has, the CPU writes a new 16-bit tempo to the MIB, so 
that MIDI notes may be appropriately scheduled for output. 

10 The functions described above and embodied in the 

appended software require mention of several additional 
points. First, if a guest arrests all movement during the 
playing of audio data, the software must be capable of 
freezing the computer's continued correlation of scalars, 

15 to avoid skewing the two hundred position circular buffer, 

or history, described above. In the Guest Controlled 
Orchestra, the computer's CPU does not perform the 
correlation steps if the number of changed pixels for a 
given image frame is less than or equal to four. Thus, 

20 the circular buffer will not be updated until the guest 

once again continues movement. Also, the computer's CPU 
will arrest the playing of music by the MIB by loading the 
MIB's tempo register with a minimum tempo, indicating that 
music is to only be slowly output by the MIB to the MIDI 

25 synthesizer. 

Also, it is noted that the algorithm utilized in the 
preferred embodiment for deriving the scalar from the 
centroids for each frame is susceptible to a minimum tempo 
if the guest engages in particular movement, i.e., 
30 movement of arms that produces a constant scalar as a 

result of the scalar computation algorithm. It is 
expected that those skilled in the art can implement an 
alternative algorithm that avoids this result without 
departing from the principles of the invention. 
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In addition, the computer' s RAM must store the total 
number of beats of the chosen musical score. The system 
10 preferably uses the song "Under The Sea" from the 
Disney movie "The Little Mermaid," in a theme park 
5 setting. The computer's CPU determines from this 

information the number of MIDI commands in the audio 
sequence, and thereby determines when its pointer 
references the last note in the song. It formats and 
accordingly loads this final command to the MIB's 
10 register, and terminates the main program loop, as 

indicated in FIG. 3. 

The last step of the main program loop, as shown in 
FIG. 3, is to synchronize the video disk player 30 with 
the music generated with by the audio system. The GPU 

15 determines, from comparison of the tempo to the score's 

ideal tempo, the rate at which the video disk player 30 is 
told to advance frames. As video frame advance is 
directed by the CPU once every thirtieth of a second, the 
computer will normally direct the video disk player to 

20 advance one frame. To the extent that the comparison of 

the guest's tempo and the ideal tempo are not the same, 
i.e., their ratio is not an integer, the CPU accumulates 
any excess to be applied to the next program cycle. Thus, 
if the guest's tempo is slower than the ideal tempo, the 

25 computer will not instruct the video disk player 30 to 

advance, but will accumulate the guest's beats per minute 
for addition to the subsequent tempo calculation during 
the next video frame advance step. In the Guest 
Controlled Orchestra, the video disk player 3 0 does not 

30 communicate to the computer 20, and hence, the program 

must instruct the CPU to keep independent tally of the 
progress of the video frame display sequence. 

The disk player used in the system is a "TQ-3032 F" 
optical disk player, available from Panasonic Industrial 
35 Company of Secaucus, New Jersey, and, as mentioned, is 
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coupled to a port of the computer via the RS-232 
connector. Each time the video frame advance is called, 
the computer will either not instruct the video player or 
it will instruct the video disk player to advance from 
5 one-to-five frames. The disk player automatically 

provides a video rate output of the frame it is instructed 
to display, and continues displaying that frame, thirty 
times per second, until instructed to advance by the 
computer. 

10 The second of the CPU's interrupts is utilized to 

load data to the RS-232 port for communication to the 
video disk player. Every one- thirtieth of a second, the 
CPU will either not command the optical disk player or 
will format the 1-5 frame advance commands, as discussed. 

15 The second interrupt is repeatedly called to load 

successive individual bytes of this command into an output 
buffer of the RS-232 port until there are no remaining 
bytes of the command. After a single byte is loaded, the 
interrupt is subsequently reset, and the loaded 

20 information transmitted bit -by-bit until the output buffer 

is once again empty. The interrupt will be triggered for 
loading a remaining byte into the output buffer as long as 
any bytes in the video command remain. 

A further refinement of the system 10 within the 
25 scope of the invention would be to implement communication 

from the video disk player to the computer to indicate 
frame number. In this manner, the computer 20 could 
directly read the frame number, instead of keeping 
independent tally of the frame, as mentioned above. There 
30 are commercially available video disk players which have 

an output port to provide this connection. 

Those skilled in the art will observe that numerous 
changes may be made to the Guest Controlled Orchestra 
without departing the principles of the current invention. 
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For example , implementations may be easily devised which 
correlate values separately for each window, or which 
simply determine periodicity by analyzing when the 
scalar' s path of movement, or magnitude with respect to 
5 time, transgresses a predefined value. 

Having thus described several exemplary embodiments 
of the invention, it will be apparent that various 
alterations, modifications and improvements will readily 
occur to those skilled in the art. Such alterations, 

10 modifications, and improvements, though not expressly 

described above, are nonetheless intended and implied to 
be within the spirit and scope of the invention. 
Accordingly, the foregoing discussion is intended to 
illustrative only; the invention is limited and defined 

15 only by the following claims and equivalents thereto. 
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APPENDICES 

Appendix A is a "make" file for the programs and 
routines found in appendices B - G and enables usage 
directly from the computer's disk operating system. 



5 Appendix B is a software listing in "C" language" of 

the main program loop. 

Appendix C is a machine language listing entitled 
"COMM. ASM" , that handles low level control routines. 

Appendix D is a machine language listing entitled 
10 "GCOJJTIL.ASM" that includes various utility routines. 

Appendix E is a machine language listing containing 
MIB interface routines. 



Appendix F is a machine language listing entitled 
"SCREEN. ASM" that includes routines for generating 9 X 16 
15 dot matrix characters and controlling display screen 

functions . 

Appendix G is a machine language listing entitled 
"VIDEO. ASM" that includes image processing routines, 
routines for computes centroids and scalars, and routines 
20 for correlating and analyzing the scalars to provide tempo 

information. 



WO 93/22762 



PCT/US93/03667 



- 38 - 



0B\GO0.OBJ: OGDEXGCO.C 
tec -c -C 



GB\MP04O1G.GBJ : CQD&MIFMQ1G.ASK 

/mac codrt^xUOlg, db\jnpaiQlg; 



ob\j 



C3B\0QM1.0BJ: O0DE\OGMf*ASM 

/« cocJeXeonm, ob\ 



GBVGOMTTZL.aBJ: OODBXGOOJDTtt.JiSM 

tasm /mx cod»\gcqjtxtiX r bbXgeojotil; 

QBWIDEO.OBJ: CODE \VIDEO. ASM 

/ax cocte\vldeo, ob\v±deo; 



^^ E ™ X 58- QB f 0B\MPU401G.OBJ CmSCREEN.GBJ GB\CG*i.OBJ 
OB\GCO OTIL^OBJ QBWXDEO.OBJ wxwwra.uou 

Etink ecode\m.lnk 
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\T0RBO\C0S + 
OB\GCO + 
OBVHPU401G + 
OB\SCREEN + 
OBXCOMM + 
OB\GCO_UTIL + 
OBWIDEO 

GCO, GCO, \TURBO\CS; 
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WDI Blowpop Controlled Orchestra 
with videodisc visual accompaniment 



finclude "MUSIC. H" 

#define HALT_PERIOD 450 
#define MIN_DELTA_PERIOD 10 
#define MINIMUM__COUNT 5 
#define BAND GAP 4 



#define TICK_PER_FRAME 26 

/* ceiling (192 tick/beat * 240 beat/min / 1800 frame/min) */ 

/+ 192*120/900 reduces to 128/5 */ 
^define TPFJJUMERATOR 128 
#define TPF_DENOMINATOR 5 
^define I?F_REMAINDER 3 

/* TPF_NUMERATOR - (TPF * TPF DENOMINATOR) */ 



int count; 

header_rec 
song_Jieader; 

exuern note 

far ~note_Q_ptr [ ] ; 

note 

far ~tracJcmem[8] ; 

meas__rec 
meas [30 ] ; 

zrac)c_tiine[8] [MAX_TIME_CHANGES] ; 

long 
unit_start; 

int 

cempo_override, 
velocity_override_0, 
velocity_override_l , 
veiocity_pverride_2,. 
velocity_override__3 > 
velocity_override_4 r 
velocity_override_5 , 
velocity_override_6, 
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velocity_override_7 , 

max_cent, 
min_cent, 

l_x_cent = 0, 
l_y_cent = 0, 
l_count, 

r_x_cent = 0, 
r — y_cent = 0, 
recount, 

l_volume, 
r_yolume, 
m_yolume, 

centroid, 

new_period, 
oldjperiod, 
rmning_period, 
periods- 
unsigned int 

avai ljnemjsi z e , 
track_mem_si ze ; 

char 

*file_names[] = 

{ 

M S0NGS\\UT_SEA1 . SNG" , 
" SONGS \ \EKN . SNG" , 
"SONGS \ \UT_SEA1 . SNG" , 
"SONGS \\UT_SEA1 .SNG", 
} / 

file_id[] = "MidiCAD Version 1.00 Song File", 
rising, 

active_trac]cs, 
raessage_ready, 
song_i s_playing, 

out_string[80], 

velocity_loo)cup[] = 

2 ?' 2' 2' %' 2' 2' °' l ' l ' x ' 1' 1' 1» 

2, 2, 2, 2, 2, 2, 2 2, 3, 3, 3, 3, 4, 4, 4, 4, 

17 ifi i ' „' ,2' 8 ' 8 ' 9 ' 10 ' U' 12 ' 13, 14, 15, 16, 

33' 12' X' ?2' «' 22' 24 ' 25 ' 26 ' 27 ' 28 ' 29 ' 30 ' 31 ' 32 ' 

X' S* if' IV 11' 37 ' 39 ' 40 ' 41 ' 42 ' 43 ' 44 ' 45 ' 46 ' 47 ' 
S' f? r 2' ff' !2' 60 - 62 - 64 - 66 ' 68 ' 70 - 72 - 74 - 76 ™ 

lS'ifi%?2',?!',22',22' 92, 94 ' 96 ' 9 8,100,102,104,106, 108,110, 
112, 114, 116, 118, 120, 122, 124 , 126, 127, 127, 127, 127, 127, 127, 127, 127 

J r 
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extern void far *ddsjnalloc () ; 
extern unsigned _heaplen = 1; 

int 

message_pverflow, /* used for calculating VDP commands */ 

messagetally, 

jump_jsize; 

char 

stxin gjnj ise: /* flag indicating that vdp_coinrnand isn't sent yet */ 

vdp_cannand; /* command string to be sent to video disk, player */ 

process gco video!) 

wait_f or_odd_f ield (J- ; 

l_count = get_l_centroid(&l_x_cent, Sljrjcent) ; 
recount = get_r_centroid(£r_x_cent, srjrjcent) ; 

count = (Ijcount + r_count) / 2; 

if (count >= MINIMUM COUNT) 

{ 

centroid = ( 

(l_x_cent + l_y_cent) * I__count + 
(r_x_cent + r_y_cent:) * recount 
) / count / 4; 

campute_correlation (centroid) ; 



new_j>eriod = get_period() ; 

if (abs (new_period - old_period) < MIN DELTA PERIOD) 
running_period = new_period; ~ 

if (new_period != 0) 

oldjjeriod = new_period; 

if (count >= MINIMUMJCOUNT) 
period = running_period; 
else 

period = HALT_PERIOD; 
display_corr_graphs (centroid, period) ; 



test_gco() 

{ 

while (1) 

{ 

init_gco(); 

init_graphic_display () ; 

while (! key_ready ( ) ) 
process_gco_yideo f ) ; 

if (getJceyO — ESC) 
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break; 

if (getJceyO = ESC) 
break; 

} 

clear graphics <); 

} 



play_gco() 

{ 

int 

old_tempo = 0, 
i; 
char 



downbeat; /* hold off flag before giving guest tempo control V 

ford = 0; i < 8; i++) 
{ 

note_0_ptr[i] - track_mem[i3 ; 
if ( song_header . track_on_of f [ i ] ) 
_ active_tracks |= 1 « i ; 

if (active tracks = 0) 

{ 

dispiay_error{"No Song Is Loaded, Medfly Maggot !"); 
return (0); " 

} 

unit_start = 0; 

flush_queues() ; 
update_queues ( ) ; 

init_gco() ; 

init_graphic_di splay () ; 

setup_com2(); /* activate the serial oort to the VDP */ 

( "^f 00 ^ 3 ^3\10»); /* send command to star^Lame 1900 
wnile(string_xn_use); /* wait until VPD has command */ 

set_tempo <song_header . tempo) ; 
start_play ( ) ; 

set S ^r^; ^ Q o> Wait two bars before sanding over tempo V 

S^^^T ' '* one ^terrupt per beat V 

while <song_is_jplaying) 
process_gco_video () ; 

if (downbeat) 

{ 

if taessage_ready — 64) /* 4 measures of 4 beats 2 times 2x res V 
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downbeat = 0; 
messagejready - 0; 

set_clock_tO_host (TICK_PER_FRAME) ; 

} 

} 

else if (period != 0) 
{ 

if (messagejready > 0) 

{ 

/* jump once for each clock tick */ 

interrupt_off 0 ; /* interrupt protect this operation */ 
jump_size = messagejready; 
message_ready -= jump_jsize; 
interruption ( ) ; 

/* convert jump into ticks for precision */ 
message_tally += jurap_size * TICK_PER_FRAME; 

/* add REMAINDER to tally for each DENOMINATOR counts */ 

message_overflow +— jumpjsize; 

if (message_pverflow >= TPF_DENOMINATOR) 

{ 

message_tally += TPF_REMAINDER; 
message_overf low %= TPF_DENOMINATOR; 

} 

if Cistring_in_j2se) /* previous message not sriii pending */ 



/* back out correct number of jumps & execute */ 
jump_size = message_tally / TICK_PER_FRAME> 
if ( juirojsize > 5) jump_size - 5; /* VDP jumo limit */ 

message_tally -= jump_size * TICK_PER_FRAME ; 

switch ( jumD_size) 
{ 

case 0: 

{ 

break; 

> 

case 1: 

/* command to jump forward 1 frame */ 
out_str_com2("\2aFl:\3\13\10") ; 
break; 

} 

case 2: 

t 

/* command to jumo forward 2 frames */ 
out_str_com2 ( n \2JF2: \3\13\10") ; 
break; 

} 

case 3: 

{ 

/* command to jump forward 3 frames */ 
outjstrjsomZ ( M \2JF3: \3\13\10") ; 
break; 

} 

case 4: 
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{ 

/* command to jump forward 4 frames */ 
out_strean2 (" \2JF4 : \3\13\10") ; 
break; 

} 

case 5: 

{ 

/* command to jump forward 5 frames */ 
out_str_com2( w \2JF5:\3\13\10") ; 
break; 

} 

} 



} 



if ( (tempq_override = 3600 / period) > 255) 
tempq_override = 255; 

if ^(tempo_override old_tempo) 

set_tempo <tempq_override) ; 
oldjtempo = tempo_override; 

l_yolume = l_count * 2; 
r_volume = recount * 2; 
m_yolume = count * 2; 

Ijvolume - (l_volume > 127) ? 127 : 1 volume; 
rvolume = (r_volume > 127) ? 127 : r~volume; 
m_volume = (m_volume > 127) ? 127 : nTvolume; 

velocity_override_0 = velocity_lookup[m_volume] ; 

. ^, /* vibe on both sneakers */ 

velocity_overrrde_l - velocity_lookup [rvolume] ; 

t /* bass, on Quest's left sneaker */ 

velocxty_pverride_2 = velocity_lookup[l_volume] ;" speaker / 

-, /* flute, on guest's riciht soeaJeer * 

velocity_override_3 - velocity_lookup(m_volume] ; ' speaker 

, /* drums on both speakers *■/ 

velocity_override_4 = 0; * / 

velocity_override_5 = 0; 

velocity_override_6 = 0; 

velocity_override 7=0; 

} 

update_queues ( ) ; 

if (key_ready() ) 

if(get_key() — • *) 
break; 

stop_play () ; 

while (string^in^use); /* wait to insure that buffer ±s ic/ 

tey"wS(); ; '* St ° P CQm2 interru P ts ' message is sent */ 

clear_graphics ( ) ; 
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raainC) 
{ 

int i; 

#if def FALSE 
int3(); 

setup_com2 0 ; /* activate the serial port to the VDP */ 

out:_str_ccan2C w \2SR1900:\3\13\X0*); /* send command to start at frame 1900 
while <string_in_use) ; /* wait until VBD has command */ 

int3(); 

kill_com2<); /* watch the com port closure */ 

int3(); 
exit(0) ; 
#endif 

inithereO ; 
set_graphics () ; 

init_sparkle_lut () ; 
stash_int() ; 
set_mpu_int () ; 
reset_mpu() ; 
init_mpu() ; 

avail_mem_size « get_avail_memO ; 
track_mem_size = (avail_mem_size / 8) - 1; 

for(i = 0; i < 8; i++) 

track_mem[i] = dos_malloc(track_mem_size) ; 

new_screen: 

display_centered("W D I", 0); 
dispiay_centered("G U E S T", 2); 
display_centered("C O N T R O L LED", 3); 
display_centered ( "O R C H E. S T R A", 4) ; 

display_centered( "Available Selections",, 8) ; 
display_centered("l - UNDER THE SEA'V 10); 
display_centered ( "-2 - NACHT MUSIC", 12) ; 
display_centered ( n 3 - UNDER THE SEA", 14) ; 
display_centered("4 - UNDER THE SEA", 16); 

display__centered( "Whack The Spacebar To Start, Dude", 19); 

display_centeredC"A - Align Camera T - Test Without Music", 23); 

while(l) 

{ . 

active_tracks - 0; 

switch Cget key () ) 

{ 

case ESC: 
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} 



} 



f ix_xnt ( ) ; 
set_text 0 ; 
exit(O); 

case 'A 1 : 
case 'a 1 : 

init_norm_lut ( ) ; 

align_camera ( ) ; 

init_sparkle_lut () ; 
break; 

case , T 1 : 
case 't': 

test_gco<) ; 
goto new_screen; 

case 1 1 1 : 

load_file(0); 
break; 

case f 2* : 

load_file<l); 
break; 

case '3': 

load_file (2) ; 
break; 

case 9 4 1 : 

load_file (3); 
breaks- 
case 1 1 : 

playjgcQ ( ) ; 
goto new_screen; 
} 



load_file (index) 
inr index; 

I 

int 

i, 

meas_size, 

time_size, 

byte_count, 

header_size, 

filehandle; 

if {(filehandle = _open(file_names [index] , 0) ) — -1) 

display_error( "Unable To Open File"); 
return (0); 

} 

if (_read (filehandle, out_string, sizeof (f ile_id) ) ! = sizeof (file_id) ) 
1990 Walt Diinty ImiginMring 
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goto readjsrror; 

if (strcopiout string, file id) ) 
{ 

display_error ("Invalid File") ; 
_close Cf ilehandle ) ; 
return (0); 

} 

if (_read(f ilehandle, &song_header, sizeof (song_header) ) J- sizeof (songjheader) 
goto readjerror; 

meas L size = song_header*nuia_meas_names * sizeof (measjrec) ; 

if (_read(f ilehandle, &meas[0J, meas_size) != meas_size) 
goto readjerror; 

forfi = 0; i < 8; i++) 

time_size = song_header . num_times [ij * sizeof (time_rec) ; 
if (_read(f ilehandle, &trackj:imeti] [0] , time_size) 1= time_size) 
goto readjerror; 

byte_count = song_header*track_size[i] * sizeof (note) ; 
if (farj:ead(f ilehandle, trac3cmem[i] , byte_count) != byte_count) 
goto read error; 

} 

_close (f ilehandle) ; 
return (0); 

read_error: 

dlsplayjerror ("Unable To Read File"); 
_close (f ilehandle) ; 
return (0); 

} 

init_gco() 
{ 

int i; 

wait_for_odd_f ield () ; 
init_historyjDuff ers () ; 

velocity_override_0 = 
velocity_override_l - 
velocity_pverride_2 = 
velocity_pverride_3 = 
velocity_override_4 = 
velocityjoverride_5 = 
velocity_override_S = 
velocity_pverride_7 = 32; 

ljvolume = 
r_volume « 
mjrolume = 64; 

rising = 1; 
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rain_cent = 30000; 
max_cent = 0; 

period = new_period « old_period « runningjperiod *= 0; 

init_graphic_display () 
clear_graphics ( ) ; 

display_text ("Filtered X Left Centroid-, 0, 6, 1); 
display_text ("Running Pixel Correlation", 0, 13, 1); 
init_corr_graphs ( ) ; 

display_error (string) 
char string [] ; 

{ 

display_blanks(80, 0, 24); 
display_text (string, 0, 24); 
tey_wait(); 

displayjblanks(80, 0, 24); 
return (0) ; 

} 

display_message (string) 
char string []; 
{ 

<ttsplayjblanks(80, 0, 24); 
display_text (string, 0, 24); 
return (0) ; 

} 

display_centered (string, line) 
char string []; 
int line; 

{ 

display_blanks(80, 0, line) ; 

display_text (string, (80 - strlen (string) ) / 2, line); 
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Serial Communication Module CCM4.ASM 

Includes routines for: 
Low level control for COM! or COM2 

ALL ROUTINES CONTROL THE UARTS DIRECTLY AT THE I/O PORT LEVEL 



CCML_DATA 

CQM1_IER 

CGMIJEIR 

C0M1_LCR 

C0M1_MCR 

CQM1_LSR 

CGM1_MSR 

C0M2_DATA 

C0M2_IER 

C0M2_IIR 

COM2_LCR 

C0M2_MCR 

C0M2JLSR 

COM2 MSR 



equ 3F8h 
equ C0M1 
equ COMl* 
equ C0M1~ 
equ COMl" 
equ CCMl" 
equ COCflT 

equ 2F8h 
equ COM2 
equ COM2 - 
equ COM2" 
equ COM2" 
equ COM2]] 
equ COM2* 



DATA+1 
DATA+2 
DATA+3 
DATA+4 
DATA+5 
DATA+6 



DATA+1 
DATA+2 
DATA+3 
DATA+4 
DATA+5 
DATA+6 



;for MCR (modem control reg): 
0UT2_BIT equ 000010Q0B 

RTS_3IT equ OOOOOOIOB 

DTR_BIT equ 00000001B 



;for MSR 
DCD_MASK 
RI_MASK 
DSR_MASK 
CTS MASK 



(modem status reg) 
equ 10000000B 
equ 01000000B 
equ OOIOOOOOB 
equ 00010000B 
DCD_DELTA_MASK equ 00001000B 
RI_DELTA_MASK equ 00000100B 
DSR_D£LTA_MASK equ OOOOOOIOB 
CTS_DELTA_MASK equ 00000001B 

;for LSR (line status reg) 
TXDEJMASK equ 00100000B 

FE_MASK equ 00001000B 

PE_MASK equ 00000100B 

OE_MASK equ OOOOOOIOB 

RXDA_MASK equ 00000001B 

;for IER (interrupt enable register) 
MODEM_INT_EN equ 00001000B 
RXIE_INT_EN equ 00000100B 
TXDE_INT_EN equ OOOOOOIOB 



;C0M1 transceiver data register 
;C0M1 interrupt enable register 
;C0M1 interrupt indentification reg 
;CQM1 line condition register 
;COML modem control register- 
;C0M1 line status register 
; C0M1 modem status register 

;C0M2 transciever data register 
;C0M2 interrupt enable register 
;C0M2 interrupt indentification reg 
;CQM2 line condition register 
;CQM2 modem control register 
;C0M2 line status register 
;CQM2 modem status register 



? H -> L only 



; transmit data register empty 

; frame error 

; parity error 

; overrun error (data lost) 

; receive data available 



; modem interrupt enable 

;rx interrupt enable 

;tx data empty interrupt enable 
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RXDA_INT_EN equ 00000001B 



;rx data available interrupt enable 



;for IIR (interrupt identification register) 

IBN equ 00000001B ; active low, interrupt pending flag 

;rx error condition 



RX_ERROR_CON equ 00000U0B 

RX_CHAR_AVAIL equ 00000100B 

TXD__REG__EMPTY equ 00000010B 

MODEMJCNT equ 00000000B 



;for LCR 
DLAB_jC 

;for INT14 
BAUD110 
BAUD150 
BAUD300 
BAUD 600 
BAUD1200 
BAUD2400 
BAUD4800 
BAUD9600 

PARITVNO 
PARITYODD 
PARITYEVEN 

STOP1 
STOP 2 

DATA5 
DATA6 
DATA7 
DATA 8 

; return values 
NO_ERROR 
UARTJERROR 

;8259 values 
EOI 

PORT8259 
INT MASK 



equ 0U11111B 



equ 00000000B 
equ 00100000B 
equ 01000000B 
equ 01100000B 
equ 10000000B 
equ 10100000B 
equ 11000000B 
equ 11100000B 

equ 00000000B 
equ 00001000B 
equ 00011000B 

equ O0OOOOOOB 
equ 00000100B 

equ 00000000B 
equ 00000001B 
equ 00000010B 

equ 000000113 



equ 0 
equ 1 



equ 20h 
equ 20h 
equ 21h 

.model small 

.data? 



old_int_0B dd 0 
out_string_com2 dd 0 



;rx character available 
;tx data register empty 
;modem lines interrupt 



; divisor latch access bit (mask) 



; interrupt mask register (active low) 



;seg & off to string sent by routine 



extra _string_in_use; BYTE 
.code 



void interruption () & interrupt_of f ( ) 
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Enables/Disables interrupts 



public _interrupt_of f 
_interruptjof f proc near 
clt 

_intemzptj3ff endp 

public ^interruption 
^interruption proc near 
sti 

^interruption enc£> 



void setup_com2() 

Sets Communication Parameters For COM2 
Clears receive buffer 
Installs interrupt vector 
Enables interrupt for character transmission buffer empty 



public _setup_com2 
_setup_com2 proc near 



mov dx,l ;C0M2 (for C0M1 is 0) 

mov ah, 0 

mov al,BAUD9600+PARITYNCHSTOPl+DATA8 
int 14H 

mov dx, C0M2_DATA ;Readincr the receive register clears it 

in al,cx 

mov al,0Bh ;save old IKQ3 (INT OBh) 

mov ah, 35h ;code for aet vector 

int 21h 

mov word ptr old_int_0B,bx ; stash old vector away 
mov word ptr old_int_0B [2 ] , es 

push ds ;set COM2 interrupt vector 

mov dx, seg out_char_ccsn2 
mov ds, dx 

mov dx, offset out_char_com2 

mov al,0Bh 
mov ah,25h 
int 21h 

pop ds 
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setuD com2 



cli 

mov dx,C0M2_IER 
mov al , TXDE_INT_EN 
out dx,al 

mov dx,COM2_IIR 
in al,dx 

mov dx,CQM2_MCR 
mov al,0UT2_BIT 
out dx,al 

in al , INT_MASK 
and al,11110111B 
out INT_MASK,al 

mov al/EOI 

out PORT8259,al 

sti 

ret 

endp 



; allow transmit data empty interrupts 

; clear first xmit data empty- 
enable card's interrupts 
; secret Jcnowledge 

;Get mask 

; enable IRQ 3 

;put it back 

; clock the 8259 



void kill_com2() 

Halts Communication Parameters For COM2 
Disables COM2 interrupts 
Removes interrupt vector 



_kill_com2 



public Jcill_com2 
proc near 

mov al,0 

mov dx,C0M2_MCR 

out dx,al 

cli 

in al,INT_MASK 
or al,00001000B 
out INT_MASK,al 

mov al,EOI 

out FORT8259,al 

sti 

push ds 



r clear the OUT2, RTS & DTR lines 



;Get mask 
/disable IRQ 3 
;put it back 

; clock 8259 



/restore old vector 
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Ids dx,old_int_OB 
mov al,0Bh 
mov ah,25h 
int 21h 



pop ds 
ret 

Jcill_com2 eno$> 



void _out_str_com2 (string: char *) 

[bp+4] 

sends a \0 terminated string out via com2 



public _out_str_com2 
_out_str_com2 proc near 

push bp 
mov bp, sp 



mov bx, [bp+4] ;get pointer to first character 

mov al, [bx] ; dereference once 

cmp al, 0 ;test first character 

jz out_str_com2_9 ;bif null string 

mov _string_in_jjse, 1 ;set "string in use" flag 

inc bx ; already used first character , now sec, 

mov WORD PTR out_string_com2,bx ;save updated pointer 
mov WORD PTR out_string_com2+2, ds 

mov dx,CQM2_DATA 

out dx,al ;send character , further ones with INT 

our_str_com2_9 : 

pop bp 
ret 

_out__str_ccm2 endp 

out_char_com2 

send s next character of string when TXDE interrupt occurs 
NOTE: interrupt vector routine! ! ! 

assumes only IRQ3 (INTOB) internets come from need for IX data 
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out_char_com2 proc near 



push ds 
push dx 
push bx 
pus h ax 

mov dx,C0M2_IIR ; identify interrupt source 

in al,dx 

cmp al, TXD_REG_EMPTY ;test for intended interrupt (TxDE) 

jne out_char_com2_9 ;bif not the correct kind 

mov al,EOI ; clear the 8259 
out PORT8259,al 

mov bx,seg out_string_com2 ;load character pointer a ddr ess 
mov ds,bx 

push ds ; additional save for later... 

Ids bx, out_string_cam2 ;get character pointer 

mov al, [bx] ;ge t character 

cmp al,0 
pop ds 

je out_char_com2_8 ;bif last character gone 
inc word ptr out_string_com2 ; point to next character 
mov dx,C0M2_DATA 

out dx,al ;send character, further ones with INT 

jmp out_char_com2_9 ;jump to end 



pop ax 
pop bx 
pop dx 
pop ds 

iret 



cut char com2 8: 



mov bx, seg _string_in_use 
mov ds,bx 

mov _string_in_use, 0 



; clear "string in use" flag 



out char com2 9: 



out_char_cam2 endp 



end 



1990 Walt Disney ImagixiMring 



SUBSTITUTE SHEET 



WO 93/22762 



PCT/US93/03667 



- 56 - 



Various Assembly Utility Routines UTTL.ASM 



WDI Guest Controlled Orchestra 



.model small 
.code 




_kev_ready 



nq_key : 
_key_ready 



public _key_ready 
proc near 

mov ah,l 

int 16H 
jz no__key 

mov ax,l 
ret 

mov ax,0 
ret 

encto 



_getjcey 



xs excr 



_get__key 



public _get_key 
proc near 

mov ah r 0 
int 16H 
cmp al r 0 
jz is_ext 

mov ah, 0 

ret 

endp 



_key_wait 



public _key_wait 
proc near 

mov ah, 0 
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int 16H 
ret 



_3ceyjwait 



void far *dos_malloc (unsigned int) 
pointer = dos_malloc <paragraph_count) , 



Public _get avail mem 
_y er _ av axl_jnem proc near "~ "~ 

mov bx, OFFFFH 
mov ah, 48H 
int 21fl 

mov ax,bx 
ret 



; request excessive memory 
;DOS allocate function 



;AX has largest available block 



_get_avail_mem enc^p 



^tl far ***J Bal l°c (unsigned int) 
pointer = dos_malloc <paragraph_count ) ; 



_dos_malloc 



allocjDk: 



alloc_end: 



public _dos_malloc 
proc near 

push bp 
mov bp, sp 

mov bx, [bp+4] 
mov ah,48H 
int 21H 
jnc allocjDk 

mov dx, 0 
mov ax, 0 

jmp alloc_end 



mov dx,ax 
mov ax,0 



/requested memory in paragraphs 
;DOS allocate function 



;Alloc failed, return NULL pointer 



; Alloc OK, return far pointer to 



pop bp 
ret 



_dos__malloc endtp 
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public _far_read 
_f arjread proc near 

push bp 
xnov bpf sp 



push ds 

mov bx, [bp+4 ] 
Ids dx, [bp+6] 
mov cx, [bp+10J 
mov ah, 3FH 
int 21H 

jnc read^f^oic 
mov ax, -l 

pop ds 
pop bp 
ret 



read f ok: 



_farjread endp 
end 
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MPU401 Interface Routines MPU401G.ASM 



WDI Guest Controlled Orchestra 



QUEUE_SIZE 
MAX_OFF 

EOI 

INT_CMD 
INTMASK 

DATA_PORT 
STATJPORT 

DRR 
DSR 

NOTEjON 
MAXJTIME 

START_PLAY 
STOP PLAY 



equ 12 
equ 20 

equ 20H 
equ 20H 
equ 21H 

equ 330H 
equ 331H 

equ 4 OH 
equ 80H 

equ 90H 
equ 240 

equ OAH 
equ 05H 



;max notes in each queue 
;max notes on per track 

;code for end of interrupt 

.•interrupt controller command register 

;and mask register 

;The MPU-401 10 Ports 



rThe MPU-401 Handshake Lines 



NO_REAL_TIME_OUT 

SEND_MEASURE_END_OFF 

CLOCK_TO_HOST_OFF 

CLOCK_TO_HOST_ON 

CLEAR_PLAY_COUNTERS 

SETJTIMEBASE 

SETJTEMPO 

SET_CLOCK_TO_HOST 

ACTIVATEJTRACKS 

TIMINGJ3VERFLOW 

DATA_END 

ACK 

RESET 



equ 
equ 



32H 
8CH 
equ 94H 
equ 95H 
equ 0B8H 
equ 0G8H 
equ OEOH 
equ 0E7H 
equ OECH 
equ 0F8H 
equ OFCH 
equ OFEH 
equ OFFH 



.model small 



Structure Of Raw Note Data 



note 

start_lo 

start_hi 

duration 

pitch 

velocity 



st rue 



dw 
dw 
dw 
db 
db 



Absolute, In MPU Clocks (Low Word) 

Absolute, In MPU Clocks (High Word) 

In MPU Clocks 

Midi Pitch 0 to 127 

Midi Velocity 0 to 127 
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channel 
note 



db 0 
ends 



; Midi Channel 0 to 15 



on Dtr 



on_j>tr 



For Pointer To Note On Waiting Data 



struc 



timejtq_on_lo dw 0 

time_to_onjii dw 0 

on_channel db 0 

on_note db 0 

on_velocity db 0 



ends 



off Dtr 



For Pointer To Note Off Waiting Data 



struc 



time_to_pff_lo dw 0 
rime_co_off_hi dw 0 
of fjrhannel db 0 
off_note db 0 



cff_ptr 



ends 



For Pointer To Track Queues 



★ 



queue_j)tr struc 

riming_byte db 0 

mldi^command db 0 

^idi^note db 0 

:nidi_yelocity db 0 



queue Dtr 



ends 



******************* 



Macros 
♦m**********.*,^^ 



wait for dsr 



wait_loop: 



macro 

local wait__loop 
mov dx, STATJPORT 

in al,dx 
and al,DSR 
cmp al^DSR 
je wait_loop 
endm 
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• * 



Initialized Variables 



.data 

extrn _active_tracks rbyte 
extrn _song_is_piaying rbyte 
extrn _unit_start:dword 



active^queues db 0 

cur_track db 0 

time_to_event dw 0 

target dw 0 

queue_updated dw 0 

oldjmask db 0 



extrn 

extrn 
extrn 
extrn 
extrn 
extrn 
extrn 
extrn 
extrn 



_message_ready : byte 



_velocity_ 
^velocity] 
^velocity" 
^velocity" 
^velocity" 
^velocity" 
_velocity2 
_velocity_( 



_override_0 
_override_l 
override_2 , 
override_3; 
override^ ; 
pverride_5: 
pverride__6 : 
override 7: 



:word 
:word 
iword 
word 
word 
word 
word 
word 



Jump Table For Executing MPU Commands 



mpu_jnessage s 



dw offset trackO_data_request 
dw offset trackl_data_request 
dw offset track2_data request 
dw offset tracJc3_data_request 
dw offset track4_data request 
dw offset track5_data_request 
dw offset traek6_data_request 
aw offset track7_data_request 
dw offset timing_data_overflow 
dw offset conductor data request 
dw offset undefined! ~~ 
dw offset undefined2 
dw offset all_end 
dw offset clock_to_host 
dw offset is_ack 
dw offset systemjmessage 



.data? 



running_status db ? 



Track Timers 



* 
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timer 

timerjD 
timerl 
timer2 
timer_3 
timerjj 
timer_5 
timer_6 
timer 7 



dw ? 

dd ? 
dd ? 
dd ? 
dd ? 
dd ? 
dd ? 
dd ? 
dd ? 



; Pointer to timers 



• ★ 



_note_0_ptr 

note_l_ptr 
note__2_j>tr 
note_3_j)tr 
note_4_ptr 
notej5_ptr 
note_6_ptr 
note_7_ptr 



******************* 
Pointers To the Raw Note Data 



public _note 0 ptr 
dd ? ~ 

dd ? 
dd ? 
dd ? 
dd ? 
dd ? 
dd ? 
dd ? 



********************** 



Pointers To Heads Of Track Queues 



track 0 


_head 


dw 


7 


trackJL 


Jtiead 


dw 


7 


track 2 


"head 


dw 


7 


track 3" 


"head 


dw 


7 


track 4*" 


"head 


dw 


7 


track_5~ 


head 


dw 


2 


track 6~ 


head 


dw 




track T 


head 


dw 


7 



Pointers To Tails Of Track Queues 



trackjQ 
trackJL 
trackJ2 
trackjf 
track_4 
track_5~ 
track_6~ 
track 7" 



tail 
"tail 
tail 
tail 
tail 
tail 
tail 
tail 



dw ? 
dw ? 
dw ? 
dw ? 
dw ? 
dw ? 
dw ? 
dw ? 



U***********^ ^ TracJc Queues Themselves 
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tracJc_0__ base 
track_lj>ase 
track_2_base 
track_3Jbase 
track_4Jt>ase 
track_5Jt>ase 
traek_6j9ase 
trackJMaase 
last base 



db size 
db size 
db size 
db size 
db size 
db size 
db size 
db size 
db ? 



queuejptr * 
queue_ptr * 
queuejptr * 
queue_ptr * 
queuejptr * 
queue_ptr * 
queuejptr * 
queuejptr * 



QUEUE 
QUEUE] 

queue] 
queue" 
queue] 
queue] 
queue] 
queue"" 



SIZE 
"SIZE 
"SIZE 
SIZE 
SIZE 
SIZE 
SIZE 
SIZE 



dup (?) 

dup (?) 

dup (?) 

dup (?) 

dup (?) 

dup (?) 

dup (?) 

dup (?) 



; ***"*<**•****• 

• ★ 

;* 

• * 

r 



Note On Commands Waiting To Go On Queue 



Format 

on_note:byte, time_till_on:word, on_velocity:byte, on_channel:byte 



★ 
★ 
★ 
* 



trackJ)jon 
track_l_on 
track_2_ on 
track_3_pn 
track_4__on 
track_5_pn 
track_6jDn 
track 7 on 



db size 
db size 
db size 
db size 
db size 
db size 
db size 
db size 



on_ptr 
onjptr 
onjptr 
onjptr 
onjptr 
on_ptr 
onjDtr 
on_ptr 



dup (?) 

ciup (?) 

dup (?) 

dup (?) 

dup (?) 

ciup (?) 

dup (?) 

dup (?) 



Note Off Commands Waiting To Go On Queue 
Up To MAX_OFF Notes Can Be Waiting Per Track 

Format offnote;byte, tia*_t±U_off :word, of f_channel rbyte 



track_0_off db MAXjOEF * size 

trackJL_pff db MAX_OFF * size 

track_2_off db MAXjOFF * size 

track_3_off db MAXjOFF * size 

track_4_off db MAX OFF * size 

track_5_off db MAXjOFF * size 

track_6_of f db MAXjOFF * size 

trackJ7_pf f db MAX OFF * size 



offjptr dup 
of f jptr dup 
offjptr dup 
offjptr dup 
of fjstr dup 
of f_ptr dup 
offjptr dup 
off jDtr dup 



(?) 
(?) 
(?) 
(?) 
(?) 
(?) 
(?) 
(?) 



old int Of 



.code 
dd 0 



Compute A Note And Put It In Queue 



'note on 1 wait list 
•note off « wait list 



Assumes: SI Points to 
DI Points to 

BX Points to raw note data 

ES Points to raw note seg * 



computejiote proc near 
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push ax 



; save queue pointer 



mov queue_updated, 1 

aap [si] .on_channel, OFFH ; illegal channel means no note on wait 
jz no_note_on_waiting 
jmp canpare_times 

Check For End Of Notes ********** * <hhhmhhh>w»^ 

no_note_on_waiting: 

anp byte ptr es: [bx] .channel, OFFH /illegal channel means no note 



left 



jnz get_new_note 



;if new notes are waiting, do em 



anp [di ] . of f_channel , OFFH ; illegal channel means no note off wa: 
jz endjeanpute 

jmp do_pff ;if note off is waiting, do it 



end__compute: 



jmp isjend 



;if no new notes and no offs waiting 



********* Get New Note And Put It Onto Note On Waiting List ************** 
**********.„**** Jts stQp Time Qn Note Qff List ************»**„ Wiri *, r , t 



ger_newjiote: 



ncl: 



nc2: 



mov ax r es: [bxKstart_lo 
mov dx,es: [bx] .start_hi 

push di 
mov di, timer 
sub ax, [di] 
jnc ncl 
dec dx 

sub dx, [di+2] 
pop di 

mov [si] .time_to_on_io,ax 
mov [siKtime_to_on_hi,dx 
add ax„es:[bx]. duration 
jnc nc2 
inc dx 



; Compute time to on in Long Int 



; duration +- time to on = time to off 



search_loop: 



mov cl, es: [bx] . channel 
mov [si] .on_channel,cl 

mov cl r es: [bx] .pitch 
mov [si] .on_note,cl 

mov cl,es: [bx] .velocity 

and cl,07FH * ;masJc out marking 

mov [si] .on_velocity,cl 

****** Kestructure The 'Off Wait' List *************** 
push di 
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cmp dx, [di j .timej:ojoff_hi 
ja searchjnore 

cmp ax, [di] .time_to_off lo 
. jb insertJLt ~~ 

acid di, size off_ptr 
jmp search_loop 



mov target, di 

S P inSrt°£tf aime1 ' 0FFH ;illegal channel means no note off wai 
;*********************** Find The £nd Qf The L±st 
end_loop: 

jTspS^So^ 3 " 1161 ' ° FFH ;ille ^ al channel means no note off wax 

add di, size offjptr 
jmp end__loop 



space_loop: 



************* Make Space For The New Data < 

mov cl, [di-size off_ptr] .off note 
mov [di] .off_note,cl ~ 

mov gx, [di-size offjtr] .time to off lo 
mov [di] .time_to_off_lo,cx — — — 

mov cx, [di-size offjptr} .time to off hi 
mov [ di ] . time_to_of f_hi , cx ~ ~ " 

mov cl, [di-size offjptr] .off channel 
mov [di ] . of f_channel , cl ~~ 

sub di, size offjtr 
cmp di, target 
jnz space_loop 



insert_data: 



Insert The New Data 



mov cl , es : [bx ] , channel 
mov [ di ] . of f ^channel , cl 

mov cl,es: [bx] .pitch 
mov [ di ] . of f_not e , cl 

mov [di ] . timeJ;o_of f_lo, ax 
mov [ di ] . t ime_t o_of fjii , dx 

pep di 

add fax, size note ;point to next raw nQte 
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compare_times r 

cmp [di] .off_channel, OFFH /illegal channel means no note off wa: 
jz do_on ;if no off is waiting, process the on 

mov ax, [si] .time_to_pn_hi ; which one happens first? 
cmp ax, [di] . time_to_of f_hi 
jb dojon 

mov ax, [si] . time_tojon_lo 
cmp ax, [di] . time_to_pf f_lo 
jb do_pn 

; ******************* Put A t Note off t Onto The 
do__off : 

mov ax,bx ;save raw note pointer 

POP ;get back queue pointer 

push ax ; save that note pointer 

cmp [di] .time_to_pff_hi, 0 
jnz is_overflow 

mov ax, [di] . time_to_of f_lo 
cmp ax, MAXJTIME 
jae is_overflow 

mov time_rojavent , ax 

mov [bx] .timingjbyte,al ;put time into queue 

mov al, [di] .offjiote 

mov [bx] .midi_note, al ; and note 

mov al, [di] .off_channel 
or al,NOTE_ON 

mov [bx] .midi_command,al ;and command 

mov [bx] .midi_velocity> 0 ;note_pff = note_pn with v = 0 
« Restructure The f Off Wait' List *** 
push di 



fixup_JLoop: 



mov al, [di+size of fjptr] . off_note 
mov [di] .off_note,al 

mov ax r [di+size offj>tr] . time_to_of f lo 
mov [di ] . time_tq_of f_lo, ax 

mov ax, [di+size off_ptr] . time_tq_of f hi 
mov [di ] . timejto_pf f_hi, ax 

mov al, [di+size offj)tr] .off_channel 
mov [di ] . off_channel,al 

cmp [di] .off_channel, OFFH ; illegal channel means no note off wait 
jz fixup_done 
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add di, size offjDtr 
jmp fixup_loop 



fixup_done: 

pop di 

jmp process_events 



. ******************* A , Note 0ntQ ^ Qygyg 



do on: 



mov ax,bx 
pop bx 
push ax 



;save raw note pointer 
;get back queue pointer 
;save that note pointer 



emp [si ] . time_to_onJii , 0 
jnz is_overflow 

mov ax, [si] .time_to_on_lo 
cmp ax , MAX_TIME 
jae is_overf low 

mov time_to_event, ax 

mov [bx] .timing_byte,al ;put time into queue 

mov al, [si ] .on_note 

mov [bx] .midi_note,al ;and note 

mov al, [si] .onjzhannel 
or ai,NOTE_ON 

mov [bx] .midi_command^al ;and command 

mov al, [si] .on_velocity 

mov [bx] .midi_velocity,al ;and velocity 

mov [si] .on_channel, OFFH /illegal channel means no note on waitir 
jmp process_events 



is overflow: 



mov [ bx ) . t iming_by t e , TIMING__OVERFLOW 
mov time_to_event, MAX__TIME 
jmp process_events 



is end: 



mov ax,bx 
pop bx 
push ax 



;save raw note pointer 
;get back queue pointer 
;save that note pointer 



mov [bx ] . t imingjDyt e , 0 

mov [bx] .midi_command,DATA END 



mov al,cur_track 
xor active_queues, al 
jmp end_compute_note 



;turn the queue off 



process events: 



1990 Walt Disney Imaginoering 



SUBSTITUTE SHEET 



WO 93/22762 



PCT/US93/03667 



- 68 - 



Increment Timer 



no_carry: 



mov bx, timer 

mov ax, time_jto_event ;doubleword inc timer 

add [bx] ,ax 

jnc no_jcarry 

inc word ptr [bx+2] 



Decrement All Waiting Events 

crap [si ] . on_channel, OFFH ; illegal channel means no note on waiti 
jz dec_note_offs 

sub [si] . time_to__on_lo,ax 

jnc dec_note_offs 

dec [si] . time to on hi 



dec note offs: 



cmp [di] -of f_channel, OFFH ; illegal channel means no note off wai* 
jz end_compute_note 



sub [di ] . time_to_of f_lo, ax 
jnc nc3 

dec [di] .time to off hi 
nc3; ~ 



add di, size offjtr 
jmp dec_note_offs 



end compute note: 



pop ax ; return raw note pointer in ax 

ret 



compute_note endp 



; * ^ Flush The Track Queues 

public _f lushjqueues 
_flush_queues proc near 

push di 

mov al,_active_tracks 
mov active_queues, al 

mov ax, word ptr _unit_jstart 
mov dx, word ptr _unit_start[2] 

mov word ptr timer_0, ax 
mov word ptr timer_0[2], dx 

mov word ptr timerJL, ax 
mov word ptr timer_l [2] , dx 
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mov word prr timer_2, ax 
mov word ptr timer_2[2], dx 

mov word ptr timerJ3, ax 
mov word ptr timer_3[2], dx 

mov word ptr timer_4, ax 
mov word ptr timer_4[2J , dx 

mov word ptr timer_5, ax 
mov word ptr timer_5[2], dx 

mov word ptr timer_6, ax 
mov word ptr timer_6[2], dx 

mov word ptr timer_7, ax 
mov word ptr timer_7[2], dx 

mov track_0_head, of f set dgroup : track_OJbase 
mov track_l_head, of f set dgroup :track_l_base 
mov track_2_head, of f set dgroup :track_2Jbase 
mov track_3_head, of f set dgroup : track_3_base 
mov track_4_head, of f set dgroup :track_4Jbase 
mov track_5_head, of f set dgroup : track_5_base 
mov track_6__head, of f set dgroup : track_6_Jbase 
mov trackJMiead, of f set dgroup : trackJ7_base 

mov track_JM:ail, offset dgroup : track_OJbase 
mov track_l_tail, offset dgroup : trackJLJoase 
mov track_2_tail, offset dgroup :track_2_base 
mov track_3_tail, offset dgroup :track_3 base 
mov track_4_tail, offset dgroup :track_4_base 
mov track_5_t ail, offset dgroup :track_5_base 
mov track_6_tail, offset dgroup :track_6jDase 
mov trackJ7_tail, offset dgroup : track__7_base 

mov ax, dgroup 

mov es,ax 

lea di, track_0_on 

mov al/OFFH 

mov cx, 8 * size on_ptr + 8 * MAX_OFF * size off_ptr 
rep stosb 



pop di 
_flush__queues endp 



*r****** The Track Qui °W And Add Notes If Necessary * 



public _update_queues 
japdate^queues proc near 

push si 
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push di 

queue_loop: 

mov queue_updated, 0 
- ********************* Update Track 0 Queue **************************** 

test activejqueues, 1 
jz check_track_l 

mov cur_track,l 
mov ax,track_0_tail 
mov bx, size queue_ptr 
add bx,ax 

cmp bx, offset dgroup :track_l_base ;at end of array 
jnz not_end_0 



not end 0 : 



norr end 0: 



lea bx,track_0_base ;if at end, point to start 

cmp bx,txack_0_head ;if tail + size = head, queue is full 

jz check_track_l 

lea si,track_0_on ; point to on and off wait: lists 

lea di r track_0_off 

ies bx, _nore_JD_pt:r ;and note data 

lea dx, timerjO 
mov timer, dx 

call compute_note ;with ax = queue pointer 

mov word ptr _notejO_ptr, ax ; update raw note pointer 

mov ax, track_0_tail 
add ax, size queue_ptr 

cmp ax, offset dgroup: track 1 base rat end of array 
jnz nott_end_0 

lea ax, track_0Jbase ;if at end, point to start 
mov track_0_tail, ax 
********** Update Track 1 Queue 



check track 1: 



test active__queues,2 
jz check_track_2 

mov cur_track, 2 
mov ax, track_l__tail 
mov bx,size queue_ptr 
add bx,ax 

cmp bx, offset dgroup :trackJ2_base ;at end of array 
jnz nor end 1 
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lea bx,track_ljbase ;if at end, point to start 

?chec?Sccf ad tail + Si2S = ^ qPMe " fUl1 

lea si,track_l_on ; point to on and off wait lists 
lea di,track_l_off 

les bx,note_l_ptr ;a nd note data 

lea dx, timer_l 
mov timer, dx 

call compute_note ;with ax « queue pointer 

mov word ptr note_l_ptr,ax /update raw note pointer 

mov ax f trackJL_tail 
add ax, size queue_ptr 

cmp ax, offset dgroup: track 2 base ;at end of array 
jnz nott_endJL y 

lea ax,track_l_base ;if at end f point to start 
mov track^l^tail, ax 

******** Update TracJc 2 Queue **************************** 



check_track_2 : 



not_end_2 : 



test active_queues, 4 
jz check_track_3 

mov cur__track, 4 
mov ax,track_2_tail 
mov bx,size queue ptr 
add bx,ax 

cmp bx, offset dgroup: track 3 base ;at end of array 
jnz not_end_2 1 

lea bx,track2_base ;if at end, point to start 

;lf tail + Si2e ~ ^ *~ is full 

iii S;S a cH:o?f ;point to on Md off wait lists 

les bx, note_2_ptr ;a nd note data 

lea dx, timer_2 
mov timer, dx 

call compute_note ;with „ = queue pointer 

mov word ptr note_2jDtr,ax ; update raw note pointer 
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mov ax, track_2_tail 
add ax, size queue_ptr 

cmp ax, offset dgroup: track 3 base rat end of array 
jnz nott_end_2 



nott end 2: 



lea ax,track_2_base ;if at end, point to start 
mov track__2_tail, ax 

********* Update Track 3 Queue ****************** 



check track 3 : 



not end 3: 



nott end 3: 



check track 4: 



test active_queues, 8 
jz check_track_j4 

mov oirj:rack, 8 
mov ax,track_3_tail 
mov bx,size queue_ptr 
add bx, ax 

cmp bx, offset dgroup: track 4 base ;ar end of arrav 
jnz not_end_J3 J 

lea bx,track_3Jbase ;if at end, point to start 

cmp bx,xrack_3_head ;if ta il + size = head, queue is full 

jz check_track_4 H 

lea si,track_3_pn ; point to on and off wait lists 
lea di,track_3_off 

les bx,note_3 __ptr ;and none data 

lea dx, timer_3 
mov timer, dx 

call compute_note ;with ax = oueue Dointer 

mov word prr note__3jptr, ax ;u?da-e raw nore pointer 

mov ax, track_J3_tail 
add ax, size queue_ptr 

cmp ax, offset dgroup: track 4 base rat end of arrav 
jnz nottjendJS 1 

lea ax,track_3_base ;if at end, point to start 
mov track_3_tail, ax 

Update Track 4 Queue ********* 

******************* 
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test active_queues, 10H 
jz check_track_5 

mov curJ:rack,10H 
raov ax, track_4_tail 
mov bx,size queuejptr 
add bx,ax 

cmp bx, offset dgroup :track_5_base ;at end of array 
jnz not_end_4 

lea bx,track_4_base ;if at end, point to start 

cmp bx,trackji_head ;if tail + size = head, queue is full 

jz check_track_5 

lea si„track_4_on ; point to on and off wait lists 
lea di, track_4_off 

les bx,note_4_ptr ;and note data 

lea dx, timer_4 
mov timer, dx 

call compute_note ;with ax = queue pointer 

mov word ptr note_4_ptr , ax ; update raw note pointer 

mov ax, track_4_tail 
add ax, size queue_ ptr 

cmp ax, offset dgroup: track 5 base ;at end of array 
jnz nott_end_4 1 

lea ax,track_4_base ;if at end, point to start 
mov track 4 tail, ax 



; ***************^^ update Track 5 Queue **************************** 
check_track_5 : 

test active_queues,20H 
jz check_jtrack_6 

mov cur_track, 20H 
mov ax,track_5_tail 
mov bx,size queueptr 
add bx,ax " 

cmp bx, offset dgroup: track 6 base ;at end of array 
jnz not__end_5 J 

lea bx,track_5_base ;if at end, point to start 

? c X£S^ ead ;if tail + size - head - *-« is 



not end 5: 
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lea si,track_5_pn ; point to on and off wait lists 
lea di,track_5_pff 

les bx,note_5_ptr ;and note data 

lea dx, timer_5 
mov timer,, dx 

call compute_note ;with ax = queue pointer 

mov word ptr note_J5_ptr, ax ; update raw note pointer 

mov ax, track__5_tail 
add ax, size queue_j>tr 

cmp ax, offset dgroup :track_6_base rat end of array 
jnz nott_end_5 

lea ax,track_5_base ;if at end, point to start 

nott_endJ5: 

mov track 5 tail * ax 



********************** update Track 6 Queue **************************** 
check_track_6 : 

test activejqueues, 4 OH 
jz check_track_7 

mov cur_track, 4 OH 
mov ax, track_6_tail 
mov bx, size queue_ptr 
add bx,ax 

cmp bx, offset dgroup :track_7__base ;at end of array 
jnz not_end_6 

lea bx,track_6Jt>ase ;if at end, point to start 

not_end_6: 

cmp bx,track_ejiead ;if tail + size = head, queue is full 

jz checkjtrackj? 

. lea si,trackjS_on ; point to on and off wait lists 
lea dirtrackJSJoff 

les bx, note_6_ptr ;and note data 

lea dx, timer_6 
mov timer, dx 

call compute_note rwith ax = queue pointer 

mov word ptr note_6j>tr, ax ; update raw note pointer 

mov ax, trackjEMbail 
add ax, size queue_ ptr 

cmp ax, offset dgroup : track_7_base ;at end of array 
jnz nott__end_6 
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lea ax,track_6jDase ;if at end, point to 



mov track_6_tail, ax 



start 



; ********************* Update Track 7 Queue 
check_trackJ7: 

test active_queues, 80H 
jz end_update 

mov cur_track, 80H 
mov ax, track_7_tail 
mov bx,size queue_ptr 
add bx,ax 

cmp bx, offset dgrouprlast base ;at end of array 
jnz not_end_7 



not end 7: 



r*ctt_end_7 : 
end_update : 

queues_done: 



lea bx,track_7Jbase ;if at end, point to start 

;if tail + size = head, queue is full 



cmp bx, trackJ7_head 
jz end__update 



lea si,track_7_on ;point to on and off wait lists 
lea di,trackJ7_off 

les bx,note_7_ptr ;a nd note data 

lea dx, timer_7 
mov timer, dx 

call compute_note ;with ax • queue pointer 

mov word ptr note_7_j>tr, ax ; update raw note pointer 

mov ax, track_7_tail 
add ax, size queue_ptr 

cmp ax, offset dgrouprlast base ;at end of array - 
jnz nott_endJ7 J 

lea ax,track_7_base ;if at end, point to start 
mov trackJ7_tail, ax 



cmp queue_updated, 0 
jz queues_done 
jmp queue_loop 



pop di 
pop si 
ret 



_update_queues endtp 
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Send MPU-401 a Command In BL 



sendji^ujcommand proc near 
cli 



smc__loop: 



ack__loop : 



int 



acic_received: 



mov dx,STAT_PORT 

in al,cbc 
and al,DRR 
cmp ai,DRR 
jz smc_loop 

mov al,bl 
out dx,al 



wait_fbr_dsr 

mov dx,DATA_PORT 

in al,dx 

cmp al r ACK 

jz ack_received 

call process_data 

mov dx,STAT_PORT 
jmp ack_loop 



sti 
ret 



; Prevent DSR from triggering int 

;Is MPU 401 ready to receive data 
;Send command 



rlt is not acking because it is trying 



sene^mpu_cammand endn 




Send^MPU-401 Data Byte In BL * 



send_mpu_data proc near 

mov dx, STAT PORT 



smd_loop: 



in al,dx 
and al,DRR 
cmp al,DRR 
jz smd_loop 

mov dx,DATAJ?ORT 
mov al, bl 
out dx,al 



;Is MPU 401 ready to receive data 



ret 

seno^mpu_data endp 
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Send MPU-401 a Note Pointed By SI 



send_note_data proc near 



cmp byte ptr [si] .timing_byte, TIMINGjOVEHFLOW 
jz send_overflow 

anp byte ptr [si J .midi_cammand, DATA_END 
jz send_data_end 



GCO Override note 



sndJLoqpl: 
drr_loopl : 



drr_loop2 : 



is zero vel: 



mov cx,3 

mov dx,STAT_PORT 

in al,dx 
and al f DRR 
cmp al , DRR 
je drr_loopl 

mov dx,DATA__PORT 

lodsb 

out dx,al 

loop snd__loopl 

mov dx,STAT_PORT 

in al,dx 
and al, DRR 
cmp al,DRR 
je drr_loop2 

mov dx,DATA_PORT 

lodsb 

cmp al,0 

jz is__zero_vel 

mov al,bl 

out dx,al 
ret 



; timing, note_on, pitch 



; Velocity override is in BL 



send_overf low : 
drr_loop3: 



mov dx, STAT_PORT 

in al,dx 
and al f DRR 
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sendjdata_end: 
data_end_loop : 
drr_loop4: 



cmp al,DRR 
je drr_loop3 

raov dx,DATAJPORT 

-lodsb 

out dx,al 

add si, 3 
ret 



mov cx,2 

mov dx r STAT PORT 



in al,dx 
and al,DRR 
anp al,DRR 
je drr_loop4 

mov dx, DATA_PORT 

lodsb 

out dx,al 

loop data_jend__Ioop 

add si, 2 
ret 



; timing, data end 



;and data end message 



send note data endo 



Some Initializing Stuff 



_resetjnpu 



_reset__mpu 



public _reset_mpu 
proc near 

mov bl, RESET * 

call sendjnpu_command 

ret 
endp 



public _init_mpu 
__init_mpu proc near 

mov bl, SET_TIMEBASE 
call send mpu command 

mov bl, CLOCK_TO_HOST_OFF 
call senc^npujcommand 

mov bl , SEND_MEASDRE__ENp__OFF 
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_init__mpu 



call send_mpu_coniraand 

mov bl , NO_PEAL_TTME jOUT 
call s end_mpu_command 

ret 

endp 



public _stop__clock_to_host 
_stop_clock_to_host proc near 

mov bl, CLOCK_TO_HOST_OFF 
call send_mpu_command 

ret 

jstop_clock_to_host endp 



Start Playing Notes In The Play Queue 



public _start_play 
_start_play proc near 

mov _song__is_playing, 1 

mov bl, ACTIVATE_TRACKS 
call send__mpu_command 

mov bl, _active_tracks 
call send_mpu_data 

mov bl, CLEAR_PIAY_COUNTERS 
call send_mpu_command 

mov bl, STARTJ>LAY 
call send_mpu_command 

ret 

_start_play endp 



public _clear_play counters 
_clear_j>lay_counters proc near 

mov bl, CLEAR_PIAY_COUNTERS 
call send_mpu_coinmand 
ret 

^clear^play^counters endp 
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Stop Playing 



_stop_play 



_stop_play 



set remDo 



set temno 



stash int 



stash int 



public _stop_j>lay 
proc near 

mov bl, STOP_PIAY 
call sendjpnpu_cc*nraand 

ret 

endp 




public _set_tempo 
proc near 

push bp 
mov bp r sp 

mov bl, SETJEEMPO 
call sendjnpu^cornmand 

mov bl, [bp+4] 
call sendmpujdata 

pop bp 
ret 

endp 

Save Old IRQ 7, int OF Vector 



public _jstash_int 
proc near 



***************************** 



;code for get vector 



mov al,0FH 
mov ah,35h 
int 21h 

mov word ptr old_intJ3F,bx ; stash old vector away 
mov word ptr old_int_0F [2] , es 

ret 

endo 




public _set_mpu_int 
_set_mpu_int proc near 
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push ds 
push cs 
pop ds 

mov al,0FH 
mov ah,25h 

mov dx, offset mpu int 
int 21h 

pop ds 

in al , INT__MASK 
mov oldjnask,al 
and al ,01111111B 
out INT MASK, al 



;Get mask 

; enable IRQ 7 
;put it back 



ret 

_set_mpu_int endp 



^ix int 



Restore Int OF Vector To Original State 



public _fix_int 
proc near 

push ds 

mov dx,word ptr old_int_0F 
mov ax, word ptr old_int~0F[2] 
mov ds,ax 

mov al,0FH 
mov ah,25h 
int 21h 
pop ds 



fix int 



mov al,old_mask 
out INT_MASK,al 

ret 

encro 



;put it back 



The Int For MPU-401 Operation 



mpu_int 



proc near 

push ax 
push bx 
push cx 
push dx 
push si 
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is midi: 



rapu_int 



push ds 

mov ax r dgroup 

mov ds,ax 

push es 

mov dx,STAT_POR£ 
in al,dx 
and al,DSR 
cmp al,DSR 
jnz is_midi 

pop es 
pop ds 
pop si 
pop dx 
pop cx 
pop bx 
pop ax 

jmp old_int_0F 



mov dx, DATA_PORT 
in al,dx 

call process_data 

mov al r EOI 
out INTjCMD^al 

pop es 
pop ds 
pop si 
pop dx 
pop cx 
pop bx 
pop ax 
iret 

endp 



;Xs data ready 

;if no data from MPU-401, try old int 



r reset master interrupt controller 



process__data proc near 



amp al, OEFH 

ja. is_mpu_message 

jmp end_int 



;is it a timing byte 



is_jmpujiiessage : 



mov bl r al 
and bl,llllB 
xor bh,bh 
shl bx,l 

jmp mpu_messages [bx] 
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trackO_dat a_request : ; FO 

mov si, trackJMiead 
raov bl, byte ptr _velocity_override_0 
call send_note_data 
cmp si, offset dgroup :track__l_base 
jl track_0_end 
lea si,track_0_base 

track_0_end: 

mov trackJMiead, si 
jmp end_int 

trackl_data_request : ; fi 

mov si, track_l_head 
mov bl, byte ptr _yelocity_override 1 
call sendjnote_data 
cmp si, offset dgroup :track_2_base 
jl track_l__end 
lea si, track 1 base 

track_l_end: 

mov track_l_head, si 
jmp end_int 

track2_data_request : ; F2 

mov si, track_2_head 

mov bl, byte ptr _velocity__override 2 

call send_note_data 

cmp si, offset dgroup: track 3 base 

jl track_2_end 

lea si, track 2 base 

track_2_end: 

mov track_2_head, si 
jmp end_int 

track3_jjata_request : ; ^3 

mov si, track_ 3_head 
mov bl, byte ptr _velocity_override 3 
call send_note_data "~ "~ 

cmp si, offset dgroup: track 4 base 
jl track_3_end ~~ 
lea si, track 3 base 

track_3_end: 

mov track_3_head, si 
jmp end_int 

track4_data_request : . F4 

mov si, track_4_head 

mov bl, byte ptr _velocity_override 4 
call send_note_data ~ 
aap si, offset dgroup :track_5 base 
jl track_4_end 
lea si, track 4 base 

track_4_end: 

mov track_4__head, si 
jmp endJLnt 

track5_data_request : ; fs 

mov si, track 5 head 
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track 5 end: 



mov bl, byte ptr _velocity_override_5 

call send^notejdata 

amp si, offset dgroup : tr ack_6_base 

jl trackJSjend 

lea si,track_5_J?ase 

mov track_5__head, si 
jmp end_int 



track6_data_request: ;F6 
mov si, track_6__head 
mov bl, byte ptr jvelocityjDverride_6 
call send_notejdata 
camp si, offset dgroup : track_7_base 
jl trackjSjsnd 
lea si, track_6 _base 

track_6_end: 

mov track_6_head, si 
jmp endjlnt 

track7_dat a_request : ; F7 

mov si, track_7_head 
mov bl, byte ptr _velocity_override_7 
call send_note_data 

cmp si, offset dgroup rlastjbase ;at end of array 
jl track_7_end 
lea si, track 7 base 



track 7 end: 



mov track_7_head, si 
jmp end_int 



timing_dat a_pverf low : ; ps 

jmp end_int 

conduct or_data_request : ; F9 

jmp endjlnt 

undefinedl: ; fa 
jmp end_int 

undefinedZ: ;FB 
jmp end_int 

all_end: ;FC 
mov _song_is_playing, 0 
jmp endjlnt 

clock_to_host: ; fd 
inc _message_ready 
jmp end_int 



is ack: 



jmp end_int 



;FE 



system_message: ; ff 
jmp end_int 

song_position: 
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midi_start: 
midi_stop: 
midi continue: 



jmp end_int 
jmp end_int 
jmp end_int 
jmp endjLnt 



system_exclusive : 

jmp end_int 

not_used: 

jmp end_int 

end_int : 

ret 

process_data endp 



********** Set the Clock To Host 

public _set clock to host 
_set_clock_to_host proc near ~" ~ 

push bp 
mov bp, sp 

mov bl, SET_CLOCK_TO__HOST 
call sendjmpu^command 

mov bl, [bp+4] 
call send_jnpu_data 

mov bl, CLOCKJTO_HOST_ON 
call send_mpu__command 

pop bp 
ret 

_set_clock_to_host endp 
end 
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9 x ±6 Character Generator And 
Screen Control Functions SCBEEtf.ASM 
Hercules Monochrome Graphics 

WDI Guest Controlled Orchestra 



INDEX_6845 equ 3B4H 

CONTROL_6845 equ 3B8H 

N0J50TS equ 00000000B 

ALLJDOTS equ 11111111B 

.model small 
.data 



chars db 


OOH, 


db 


OOH, 


db 


OOH, 


db 


OOH, 


db 


OOH, 


db 


OOH, 


db 


OOH, 


db 


OOH, 


db 


OFFH, 1 


db 


OOH, 



;10h 



© 



£ °^,0FFH f OFFH,OFFH,0C3H, 99H, OBDH, OBDH, 99H, 0C3H, OFFH, OFFH^ OFFH^ OFFH 

t „°»' Sn^ 1™' ¥*' 0DH ' 19H ' 3CH ' 66H ' 66H ' 66H ' 3CH, OOH OOH, 0OH 

£ 22*' f CH ' 66H ' 66H '66H, 3CH, 18H, 7EH, 18H, 18H, OOH OOH, OOH 

£ S' l TRr 33H ' 3FH ' 30H ' 30H ' 30H ' 70H,OFOH,OEOH, OOH OOH OOH 

^ 22-' 222' TP' 63H ' 7FH ' 63H ' 63H ' 63H ' 67H,0E7H,0ESH,0C0H, OOH OOH 

ob OOH, OOH, 18H, 18H,0DBH, 3CH,0E7H, 3CH,0DBH, 18H, 18H, OOH, OOH, OOH 

db OOH, OOH, 40H, 60H, 70H, 7CH, 7FH, 7CH, 70H, 60H, 40H,'00H, OOH, OOH 

^ nn»' 22*' ?£' 2*' ° 7H ' 1FH ' 7FH ' 1FH ' 07H ' 03H ' 01H < OOH, OOH, OOH 

^ 222' 22*' H H ' 3CH ' ?£H ' 18H ' 18H ' 18H ' 7EH ' 3CH < 18H, OOH, OOH OOH 

£ Su' 222' 33H ' 33H ' 33H ' 33H ' 33H * 33H ' OOH, 33H, 33H, OOH OOH OOH 

£ 222' 2*2' 7FH,0DBH,0DBH,0DBH, 7BH, 1BH, 1BH, 1BH, 1BH, OOH OOH OOH 

^ u' nn2' ™2' ^2*' 1CH ' 36H ' 63H ' 63H ' 36H ' 1CH ' 06H ' "H, OOH 

^ 222' °° H ' ° 0H ' ° 0H ' 00H ' 00H ' 00H ' 7FH ' 7FH, 7FH, OOH, OOH OOH 

^ 222' ^' " H ' 3CH ' 7EH ' 18H > 18H ' 18H r 7EH, 3CH, 18H, 7EH, OOH OOH 

^ OOH, OOH, 18H, 3CH, 7EH, 18H, 18H, 18H, 18H, 18H, 18H, OOH, OOH OOH 

<? OOH, OOH, 18H, 18H, 18H, 18H, 18H, 18H, 7EH, 3CH, 18H, OOH, OOH OOH 

cm OOH, OOH, OOH, OOH, OCH, 06H, 7FH, 06H, OCH, OOH, OOH, OOH OOH OOH 

^ 222' 222' 22*' °H H ' 18H ' 30H ' 7FH ' 30H ' 18H « OOH, OOH, OOH, OOH OOH 
db OOH, OOH, OOH, OOH, OOH, SOH, 60H, 60H, 7FH, OOH, OOH, OOH, OOH, OOH 

^ 222' 22*' 00H ' ° 0H ' 24H ' 66il '0FFK, 66H, 24H, OOH, OOH, OOH OOH, OOH 
OOH, OOH, OOH, 08H, 1CH, 1CH, 3EH, 3EH, 7FH, 7FH, OOH, OOH, OOH OOH 
db OOH, OOH, OOH, 7FH, 7FH, 3EH, 3EH, 1CH, 1CH, 08H, OOH, OOH, OOH, OOH 

db OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH 
db OOH, OOH, 18H, 3CH, 3CH, 3CH, 18H, 18H, OOH, 18H, 18H, OOH, OOH, OOH 
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db OOH, 63H, 63H, 63H, 22H, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, 00H 

db OOH, OOH, 36H, 36H, 7FH, 36H, 36H, 36H f 7FH, 36H, 36H, OOH, OOH, OOH 

db OCH, OCH, 3EH, 63H, 61H, 60H, 3EH, 03H, 43H, 63H, 3EH, OCH, OCH, OOH 

db OOH, OOH, OOH, OOH, 61H, 63H, 06H, OCH, 18H, 33H, 63H, OOH, OOH, OOH 

db OOH, OOH, 1CH, 36H, 36H, 1CH, 3BH, 6EH, 66H, 66H, 3BH, OOH, OOH, OOH 

db OOH, 30H, 30H, 30H, 60H, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH 

db OOH, OOH, OCH, 18H, 30H, 30H, 30H, 30H, 30H, 18H, OCH, OOH, OOH, OOH 

db OOH, OOH, 18H, OCH, 06H, 06H, 06H, 06H, 06H, OCH, 18H, OOH, OOH, OOH 

db OOH, OOH, OOH, OOH, 66H, 3CH, OFFH, 3CH, 66H, OOH, OOH, OOH, OOH, OOH 

db OOH, OOH, OOH, 18H, 18H, 18H,0FFH, 18H, 18H, 18H, OOH, OOH, OOH, OOH 

db OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, 18H, 18H, 18H, 30H, OOH, OOH 

db OOH, OOH, OOH, OOH, OOH, OOH,OFFH, OOH, OOH, OOH, OOH, OOH, OOH, OOH 

db OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, 18H, 18H, OOH, OOH, OOH 

db OOH, OOH, 01H, 03H, 06H, OCH, 18H, 30H, 60H, 40H, OOH, OOH, OOH, OOH 

db OOH, OOH, 3EH, 63H, 67H, 6FH, 7BH, 73H, 63H, 63H, 3EH, OOH, OOH, OOH 

db OOH, OOH, OCH, 1CH, 3CH, OCH, OCH, OCH, OCH, OCH, 3FH, OOH, OOH, OOH 

db OOH, OOH, 3EH, 63H, 03H, 06H, OCH, 18H, 30H, 63H, 7FH, OOH, OOH, OOH 

db OOH, OOH, 3EH, 63H, 03H, 03H, 1EH, 03H, 03H, 63H, 3EH, OOH, OOH, OOH 

db OOH, OOH, 06H, OEH, 1EH, 36H, 66H, 7FH, 06H, 06H, OFH, OOH, OOH, OOH 

db OOH, OOH, 7FH, 60H, 60H, 60H, 7EH, 03H, 03H, 63H, 3EH, OOH, OOH, OOH 

db OOH, OOH, 1CH, 30H, 60H, 60H, 7EH, 63H, 63H, 63H, 3EH, OOH, OOH, OOH 

db OOH, OOH, 7FH, 63H, 03H, 06H, OCH, 18H, 18H, 18H, 18H, OOH, OOH, OOH 

db OOH, OOH, 3EH, 63H, 63H, 63H, 3EH, 63H, 63H, 63H, 3EH, OOH, OOH, OOH 

db OOH, OOH, 3EH, 63H, 63H, 63H, 3FH, 03H, 03H, 06H, 3CH, OOH, OOH, OOH 

db OOH, OOH, OOH, 18H, 18H, OOH, OOH, OOH, 18H, 18H, OOH, OOH, OOH, OOH 

db OOH, OOH, OOH, 18H, 18H, OOH, OOH, OOH, 18H, 18H, 30H, OOH, OOH, OOH 

db OOH, OOH, 06H, OCH, 18H, 30H, 60H, 30H, 18H, OCH, 06H, OOH, OOH, OOH 

db OOH, OOH, OOH, OOH, OOH, 7EH, OOH, OOH, 7EH, OOH, OOH, OOH, OOH, OOH 

db OOH, OOH, 60H, 30H, 18H, OCH, 06H, OCH, 18H, 30H, 60H, OOH, OOH, OOH 

db OOH, OOH, 3EH, 63H, 63H, 06H, OCH, OCH, OOH, OCH, OCH, OOH, OOH, OOH 

db OOH, OOH, 3EH, 63H, 63H, 6FH, 6FH, 6FH, 6EH, 60H, 3EH, OOH, OOH, OOH 

db OOH, OOH, 08H, 1CH, 36H, 63H, 63H, 7FH, 63H, 63H, 63H, OOH, OOH, OOH 

db OOH, OOH, 7EH, 33H, 33H, 33H, 3EH, 33H, 33H, 33H, 7EH, OOH, OOH, OOH 

db OOH, OOH, 1EH, 33H, 61H, 60H, 60H, 60H, 61H, 33H, 1EH, OOH, OOH, OOH 

db OOH, OOH, 7CH, 36H, 33H, 33H, 33H, 33H, 33H, 36H, 7CH, OOH, OOH, OOH 

db OOH, OOH, 7FH, 33H, 31H, 34H, 3CH, 34H, 31H, 33H, 7FH, OOH, OOH, OOH 

db OOH, OOH, 7FH, 33H, 31H, 34H, 3CH, 34H, 30H, 30H, 78H; OOH, OOH, OOH 

db OOH, OOH, 1EH, 33H, 61H, 60H, 60H, 6FH, 63H, 33H, 1DH, OOH, OOH, OOH 

db OOH, OOH, 63H, 63H, 63H, 63H, 7FH, 63H, 63H, 63H, 63H, OOH, OOH, OOH 

db OOH, OOH, 3CK, 18H, 18H, 18H, 18H, 18H, 18H, 18H, 3CH, OOH, OOH, OOH 

db OOH, OOH, OFH, 06H, 06H, 06H, 06H, 06H, 66H, 66H, 3CH, OOH, OOH, OOH 

db OOH, OOH, 73H, 33H, 36H, 36H, 3CH, 36H, 36H, 33H, 73H, OOH, OOH, OOH 

db OOH, OOH, 78H, 30H, 30H, 30H, 30H, 30H, 31H, 33H, 7FH, OOH, OOH, OOH 

db OOH, 0OH,0C3H,0E7H,0FFH,0DBH,0C3H,0C3H,0C3H,0C3H, 0C3H, OOH, OOH, OOH 

db OOH, OOH, 63H, 73H, 7BH, 7FH, 6FH, 67H, 63H, 63H, 63H, OOH, OOH, OOH 

db OOH, OOH, 1CH, 36H, 63H, 63H, 63H, 63H, 63H, 36H, 1CH, OOH, OOH, OOH 

db OOH, OOH, 7EH, 33H, 33H, 33H, 3EH, 30H, 30H, 30H, 78H, OOH, OOH, OOH ; 

db OOH, OOH, 3EH, 63H, 63H, 63H, 63H, 6BH, 6FH, 3EH, 06H, 07H, OOH, OOH 

db OOH, OOH, 7EH, 33H, 33H, 33H, 3EH, 36H, 33H, 33H, 73H, OOH, OOH, OOH 

db OOH, OOH, 3EH, 63H, 63H, 30H, 1CH, 06H, 63H, 63H, 3EH, OOH, OOH, OOH 

db OOH, OOH, OFFH, ODBH, 99H, 18H, 18H, 18H, 18H, 18H, 3CH, OOH, OOH, OOH 

db OOH, OOH, 63H, 63H, 63H, 63H, 63H, 63H, 63H, 63H, 3EH, OOH, OOH, OOH 

db OOH, OOH, 0C3H, 0C3H, 0C3H, 0C3H, 0C3H, 0C3H, 66H, 3CK, 18H, OOH, OOH, OOH 

db OOH, OOH, 0C3H, 0C3H, 0C3H, 0C3H, ODBH, ODBH, OFFH, 66H, 66H, OOH, OOH, OOH 

db OOH, OOH, 0C3H, 0C3H, 66H, 3CH, X8H, 3CH, 66H, 0C3H, 0C3H, OOH, OOH, OOH 
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db OOH, QOH,0C3H,OC3H,OC3H, 66H, 3CH, 18H, 18H, 18H, 3CH, 00H, OOH, OOH 

db OOH, 00H,0FFH,0C3H, 86H, OCH, 18H, 30H, 61H, 0C3H, OFFH, OOH, OOH, OOH 

db OOH, OOH, 3CH, 30H, 30H, 30H, 30H, 30H, 30H, 30H, 3CH, OOH, OOH, OOH 

db OOH, OOH, 40H, 60H, 70H, 38H, 1CH, OEH, 07H, 03H, 01H, OOH, OOH, OOH 

db OOH, OOH, 3CH, OCH, OCH, OCH, OCH, OCH, OCH, OCH, 3CH, OOH, OOH, OOH 

db 08H, 1CH, 36H, 63H, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH 

db OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OFFH, OOH 

db 18H, 18H, OCH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH, OOH 

db OOH, OOH, OOH, OOH, OOH, 3CH, 06H, 3EH, 66H, 66H, 3BH, OOH, OOH, OOH 

db OOH, OOH, 70H, 30H, 30H, 3CH, 36H, 33H, 33H, 33H, 6EH, OOH, OOH, OOH 

db OOH, OOH, OOH, OOH, OOH, 3EH, 63H, 60H, 60H, 63H, 3EH, OOH, OOH, OOH 

db OOH, OOH, OEH, 06H, 06H, 1EH, 36H, 66H, 66H, 66H, 3BH, OOH, OOH, OOH 

db OOH, OOH, OOH, OOH, OOH, 3EH, 63H, 7FH, 60H, 63H, 3EH, OOh' OOH OOH 

db OOH, OOH, 1CH, 36H, 32H, 30H, 7CH, 30H, 30H, 30H, 78H, OOH, OOH OOH 

db OOH, OOH, OOH, OOH, OOH, 3BH, 66H, 66H, 66H, 3EH, 06H, 66H, 3CH, OOH 

db OOH, OOH, 70H, 30H, 30H, 36H, 3BH, 33H, 33H, 33H, 73H, OOH, OOH, OOH 

db OOH, OOH, OCH, OCH, OOH, 1CH, OCH, OCH, OCH, OCH, 1EH, OOH, OOH, OOH 

^ 222' «« H ' °,£ H ' ° 6H ' °° H ' 0EH ' 06H ' 06H ' 06H ' °6H, 6B» r 66H, 3CH, OOH 
db OOH, OOH, 70H, 30H, 30H, 33H, 36H, 3CH, 36H, 33H, 73H, OOH OOH, OOH 

^ 22 H ' 22 H ' 1CH ' ° CH ' 0CH ' 0CH ' 0CH ' 0CH ' 0CH ' 0CH ' 1EH, OOH, OOH, OOH 
db OOH, OOH, OOH, OOH, OOH, 0E6H, OFFH, ODBH, ODBH, ODBH, ODBH, OOH, OOH, OOH 

^ 222' 22*' ° 0H ' °° H ' 00H * 6EH ' 33H ' 33H ' 33H ' 33H ' 33H, OOH OOH OOH 
db OOH, OOH, OOH, OOH, OOH, 3EH, 63H, 63H, 638, 63H, 3EH, OOH, OOH, OOH 

2! 222' 22 H ' ° 0H ' 00H ' 00H ' 6EH ' 33H ' 33H ' 33H ' 3EH, 30H, 30H, 78H, OOH ; 
S 222' 22 H ' 2° H ' ° 0H ' °° H ' 3BH ' 66H ' 66H ' 66H ' 3EH ' OSH, 06H OFH OOH 
db OOH, OOH, OOH, OOH, OOH, 6EH, 3BH, 33H, 30H, 30H, 78H, OOH, OOH OOH 

^ S2' 222- 22 H - 00H - O0H - ^ 63H - 38H - 0EH - 63H - 3EH ooh ooh S£ 

5 222' 22 H ' 2 8H ' 18H ' 18H ' 7EH ' 18H ' 18H ' 18H ' 13H ' OEH, OOH OOH OOH 

£ 222- 222- 22 H - 22 H - 00H ' 66H - 66H ' 66H - 66H - 66H - 3BH ooh ooh ooh 

5? 222' 22 H ' 22 H ' °° H ' 00H,0C3H,0C3H,0C3H, 66H, 3CH, 18H, OOH, OOH OOH 
222' 222' 22 H ' °° H ' 0° H '0C3H,OC3H, ODBH, ODBH, OFFH, 66H, OOH, OOH, OOH 

X 222' 222- 222- 22 H - 00H ' 63H - 36H - 1CH - 1CH - 36H - 63H - ooh ooh ooh 
^ 222' 222- 22 H ' 00H ' 00H ' 63H - 63H - 63H ' 63H ' 3FH - 03H - oe H , 3 ch ooh 
^ 222' 222- 22 H - °2 H - 00H ' 7FH ' 66H ' 0CH ' 18H - 33H - 7FH - ooh ooh ooh 

222' 22 H ' ° EH ' 18H ' 18H ' 18H ' 70H ' 18H ' 18H, OEH, OOH; OOH OOH 

S 222- 222- i22' 18H - 18H - 18H ' 00H - 18H ' 18H - 18H - 18H ooh ooh Soh 

^ 222' 22 H ' 70H ' 18H ' 18H ' 18H ' OEH, 18H, 18H, 18H, 70H, -OOH OOH OOH 

£ 222' 222- n2 H - 00H ' 00H ' 00H ' Q0H - 00H ' 00H ' o° H ' ooh ooh ooh 

db OOH, OOH, OOH, OOH, 08H, 1CH, 36H, 63H, 63H, 7FH, OOH, OOH^ OOH, OOH 

herc_screen dw OBOOOH 

gtable db 35H, 2DH, 2EH, 07H 

db 5BH,02H,57H,57H 
db 02H, 03H, OOH, OOH 

ttable db 61H, 50H, 52H, OFH 

db 19H,06H,19H,19H 
db 02H, ODH, OBH, OCH 
public _init_herc 

forty_hex dw 40H 

.code 
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oldjoode db ? 

public eight 
eight db 8 

nine db 9 

fourteen db 14 

three_fifteen dw 315 



public _clear_graphics 
_clear_graphics proc near 

push di 



mov cx,4000h ; words to clear 

mov es, herc_screen 
xor di,di 
mov ax, 0 

rep stosw ; clear mem 

pop di 
ret 



_clear_graphics endp 



display J:ext (string, x f y) ; 

4 6 8 

public _display_text 
_display_text proc near 

push bp 
mov bp,sp 

mov si, [bp+4] 
mov ax,0B000H 
mov es,ax 

mov bx, offset chars 

mov ax, [bp+8] ;Get line 

and ax, 1 

jz even_line 

jmp odd_line 



even line: 



mov ax, [bp+8] ;Get line 

mul three_fifteen 
mov di,ax 

mov ax, [bp+6] ;Get col 

add di,ax 
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div eight 

mov cl, ah ; Use remainder for shifts 

xor ah, ah 
add di,ax 

c_loop: 

mov al, [si] 
cap al, 0 
jnz q_ok 
jmp c_quit 

c_o)c: 

push bx 
inc si 

mul fourteen 
add bx,ax 
xor ah, ah 

mov al, [bx] 
xor ah r ah 
ror ax, cl 
inc bx 

xor es: [di] ,ax 

mov al, [bx] 
xor ah r ah 
ror ax,cl 
inc bx 

xor es: [di+2000H] ,ax 

mov al, [bx] 
xor ah, ah 
ror ax,cl 
inc bx 

xor es: [di+4000H] , ax 

mov al, [bx] 
xor ah, ah 
ror ax,cl 
inc bx 

xor es: [di+6000H] ,ax 

mov al, [bx] 
xor ah r ah 
ror ax r cl 
inc bx 

xor es : [di+5ah ] , ax 

mov al, [bx] 
xor ah, ah 
ror ax,cl 
inc bx 

xor es: [di+205aH] ,ax 

mov al, [bx] 
xor ah r ah 
ror ax,cl 
inc bx 

xor es: [di+405aH] ,ax 
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mov al, [bx] 
xor ah, ah 
ror ax, cl 
inc bx 

xor es: [di+605afl] ,ax 

mov al, [bx] 
xor ah, ah 
ror ax,cl 
inc bx 

xor es: [di+0b4h] ,ax 

mov al, [bx] 
xor ah, ah 
ror ax,cl 
inc bx 

xor es: [di+20b4H] ,ax 

mov al, [bx] 
xor ah, ah 
ror ax,cl 
inc bx 

xor es: [di+40b4H] ,ax 

mov al, [bx] 
xor ah, ah 
ror ax,cl 
inc bx 

xor es: [di+60b4HJ,ax 

mov al, [bx] 
xor ah, ah 
ror ax, cl 
inc bx 

xor es: [di+lOeH] ,ax 

mov al, [bx] 
xor ah, ah 
ror ax,cl 
inc bx 

xor es: [di+210eH] , ax 

pop bx 
inc cl 
anp cl, 8 
jl not_8 
xor cl,cl 
inc di 

not_8 : 

inc di 
jmp c_loop 

c_quit : 

pop bp 
ret 

oddJLine : 

mov ax, [bp+8] ;Get line 
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nrul three_fifteen 
sub ax, 45 
mov di,ax 

mov ax, [bp+6] ; Get col 

add di,ax 
div eight 

mov cl,ah ; Use remainder for shifts 

xor ah r ah 
add di,ax 

oc_loop: 

mov al, [si] 
cmp al,0 
jnz oc_ok 
jmp oc_guit 

ocjok: 

push bx 
inc si 

mul fourteen 
add bx,ax 
xor ah, ah 

mov al, [bx] 
xor ah, ah 
ror ax, cl 
inc bx 

xor es: [di+4000H] r ax 

mov al, [bx] 
xor ah, ah 
ror ax,cl 
inc bx 

xor es: [di+6000H],ax 

mov al, [bx] 
xor ah, ah 
ror ax,cl 
inc bx 

xor es: [di+5ah] ,ax 

mov al, [bx] 
xor ah, ah 
ror ax, cl 
inc bx 

xor es: [di+205aH] ,ax 

mov al, [bx] 
xor ah, ah 
ror ax,cl 
inc bx 

xor es: [di+405aH] ,ax 

mov al, [bx] 
xor ah, ah 
ror ax,cl 
inc bx 

xor es : [di+605aH] , ax 
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mov al f [bx] 
xor ah, ah 
ror ax,cl 
inc bx 

xor es: [di+0b4h] ,ax 

mov al, [bx] 
xor ah, ah 
ror ax f cl 
inc bx 

xor es: [di+20b4H] ,ax 

mov al, [bx] 
xor ah, ah 
ror ax, el 
inc bx 

xor es: [di+40b4H] , ax 

mov al, [bx] 
xor ah, ah 
ror ax,cl 
inc bx 

xor es: [di+60b4HJ ,ax 

mov al, [bx] 
xor ah, ah 
ror ax, el 
inc bx 

xor es: [di+10eH],ax 

mov al, [bx] 
xor ah, ah 
ror ax, el 
inc bx 

xor es: [di+210eH] , ax 

mov al, [bx] 
xor ah, ah 
ror ax,cl 
inc bx 

xor es: [di+410eH],ax 

mov al, [bx] 
xor ah, ah 
ror ax,cl 
inc bx 

xor es: [di+610eH] ,ax 

pop bx 
inc cl 
cmp el, 8 
jl onot_8 
xor cl, cl 
inc di 

onot_8 : 

inc di 

jmp oc_loop 
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oc_quit: 



pop bp 
ret 



__display_text endp 



display_blanks (count, x, y) ; 

4 6 8 



public jdisplay_blanks 
j±Lsplay_blanks proc near 

push bp 
mov bp,sp 
push ds 

mov ax, OBOOOH 
mov ds,ax 



even b line: 



mov ax, [bp+8] 
test ax,l 
jz even_b_line 
jmp odd_b_line 



mul three_fifteen 

mov di,ax 

mov ax, [bp+6] 

add di,ax 

div eight 

mov cl,ah 

xor ah, ah 

xor ch,ch 

add di,ax 

mov bl,ALL_DOTS 
shr bl r d 
not bl 

mov ax, [bp+4] 
mul nine 
sub ax, 8 
add ax, cx 
div eight 

mov cl,al 

call clearjone 

jcxz no_loop 
mov bl, NO DOTS 



;Get line 



;Get col 



; Use remainder for shifts 



; Load all dots 

; shift for proper first dot pos 
; Invert 

; Load count of spaces 
; Times 9 for dots 

; Minus (8-shifts) 

; Divided by 8 for bytes 

; Load counter 

; Clear first partial byte 
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clear_loop: 
nq_loop: 



odd b line; 



clear_o__loop: 
no__o_loop : 



call clear_one 
loop clear_loop 

mov cl,ah 
mov bl , ALLJDOTS 
shr bl,cl 
call clear_one 

pop ds 
pop bp 
ret 

mul three_fifteen 

sub ax, 45 

mov di,ax 

mov ax, [bp+6] 

add di,ax 

div eight 

mov cl,ah 

xor ah, ah 

xor ch,ch 

add di,ax 

mov bl,ALLJDOTS 
shr bl,cl 
not bl 

mov ax, [bp+4] 
mul nine 
sub ax, 8 
add ax,cx 
div eight 

mov cl,al 

call clear_o_one 

jcxz nq_o_loop 
mov bl, NO_DOTS 

call clear_o_one 
loop clear_q_loop 

mov cl,ah 

mov bl , ALLJDOTS 

shr bl,cl 

call clear_p_one 

pop ds 
pop bp 
ret 



_di splay Jblanks endp 



; Clear all Full bytes 
; Get remainder 

; Finish off remainder of line 



;Get col 



; Use remainder for shifts 



; Load all dots 

; shift for proper first dot pos 
; Invert 

; Load count of spaces 
; Times 9 for dots 

; Minus (8-shifts) 

; Divided by 8 for bytes 

; Load counter 

; Clear first partial byte 



; Clear all Full bytes 
; Get remainder 

; Finish off remainder of line 
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clear_one proc near 

and fdi],bl 
and [di+2000H],bl 
and fdi+4000H],bl 
and [di+6000H] ,bl 
and [di+5ah],bl 
and [di+205aH],bl 
and [di+405aH] ,bl 
and [di+€Q5aH],bl 
and [di+0b4h],bl 
and [di+20b4H],bl 
and [di+40b4H] ,hl 
and [di+60b4H],bl 
and [di+10eHJ,bl 
and [di+210eH],bl 
inc di 



ret 

clear__one endp 




clear_q_one proc near 



and [di+4000H],bl 
and [di+6000H],bl 
and [di+5ah] ,bl 
and [di+205aH] r bl 
and [di+405aH],bl 
and [di+605aH] r bl 
and [di+0b4h],bl 
and [di+20b4H],bl 
and [di+40b4H] ,bl 
and [di+60b4H] ,bl 
and [di+lOeH] „bl 
and [di+210eH] r bl 
and [di+410eH],bl 
and rdi+610eH},bl 



inc di 
ret 

clear_o_one endp 

_jLnit_herc proc near 

mov dx, 3BFH 
mov al,l 
out dx,al 
ret 

_init__herc endp 
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* SET HERCULES MONOCHROME GRAPHICS MODE 



_set_graphics 



g_loop: 



timer: 



public _set_graphics 
proc near 

mov cx,2000h 

mov es, hercjscreen 

xor di,di 

mov ax,720H 

rep stosw 

lea si,gtable 

mov dx,INDEX_6845 
mov cx,12 
xor ah, ah 



mov al,ah 
out dx,al 
inc dx 
lodsb 
out dx,al 
inc ah 
dec dx 
loop g_loop 

mov dx,CONTROL_6845 
mov al, 00000010b 
out dx,al 

mov cx,4000h 

mov es, herc_screen 

xor di,di 

mov ax, 0 

rep stosw 

mov cx, 3000h 

aam 

loop timer 

mov dbc,CONTROL_6845 
mov al, 00001010b 
out dx,al 

push ds 

mov ds,forty_hex 
mov si,49H 
mov al, [si] 
mov old_mode,al 
mov byte ptr[si],6 
pop ds 

ret 



; words to clear 



; clear mem 



; 6845 index port 
; 12 params 

; starting from register zero 



; Select register 

; Point to data port 

; Get data 

; Output it to 6845 

; Next register 

; Point back to index port 



set graphics 



words to clear 



; clear mem 



; turn screen on 
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_set_graphics endp 



RESTORE MONOCHROME TEXT MODE * 



set text 



t_loop: 



timerl: 



public _set_text 
proc near 

mov cx,4000h 

mov es r hercjscreen 

xor di,di 

mov ax, 0 

rep stosw 

lea si,ttabie 

mov dx,INDEX_6845 
mov cx r 12 
xor ah r ah 



mov al,ah 
out dx,al 
inc dx 
lodsb 
out dx,al 
inc ah 
dec dx 
loop t_loop 

mov dx,CONTROL_6845 
mov al, 00000000b 
out dx,al 

mov cx,2000h 

mov es,herc_screen 

xor di,di 

mov ax r 720H 

rep stosw 

mov cx,3000h 

aam 

loop timerl 

mov dx,CONTROL_6845 
mov al, 00101000b 
out dx,al 

push ds 

mov ds; f orty_hex 
mov si r 49H 
mov al,oldmode 
mov [si],al 
pop ds 

ret 



t words to clear 



; clear mem 



; 6845 index port 
; 12 params 

; starting from register zero 



; Select register 

; Point to data port 

; Get data 

; Output it to 6845 

; Next register 

; Point back to index port 



; set text mode 



; words to clear 



; clear mem 



; turn screen on 
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_set_text endp 
end 
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Low Level Image Processing Routines VIDEO. ASM 
Regular smart centroid 



WDI Guest Controlled Orchestra 



xxxxxx 

.model small 
.data 

; Set Bank 0, 485 Lines, Start at Line 0, External Phase Lock, 
; Enable Image Acquisition 

ACQUIRE^FLAGJDN equ 00110000B 

L0W_CSR_P0RT equ 2F0H 

HIGH_CSR_PORT eou 2F1H 

L T JT_ADDRESS_PORT equ 2F2H 

?ED_LUT_DATA_PORT equ 2F3H 

3LUE_LUTJ5ATA_PORT equ 2F4H 

GHEEN_LUT_DATAJPORT equ 2F5H 

INPUT_LUT_DATA_PORT eou 2F6H 



X_INC eou 4 

Y_INC equ 6 * 512 

X_COUNT equ 40 

iJZOUXT equ 50 

INSET eou 25 ;25 

FIRST_LINE equ 50 

CPPERJEFT equ 512 * FIRSTVLINE + INSET 

UPPERJRIGHT equ 512 * FIRST_LINE + (384 - INSET) 

IONE_SI2Z eou 10 

>!IN_COUNT equ 2 



CORR__TEMPLATE__SIZE eou 30 
CORR HISTORY SIZE eou 200 



SQUARES_BUFFERJSIZE equ CORRJTEMPIATE_SIZE * CORR_HISTORYjSIZE 
WAVE_HISTORY_END equ offset waveJhistory_buffer + CORR_HISTORY_SIZE * 2 
CORR_SQUARES_END equ offset corrjsquares_buffer + SQUARES_BUFFER_SIZE * 2 
CORR_SUMSJ£ND equ offset _corr_sums_i>uffer .+ (CORR_HISTORY SIZE - 1) * 4 

BANDGAP eou 512 

DECISION_ZONE ecu 512 
MAX_GOOD_COBR equ 2048 

INITIAL_BANK equ 0 
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HI GH_THRESHOLD equ OFOh 
LOWJTHRESHOLD equ OlOh 

COUNT_FILT_SHIFT equ 2 
C0UNT_FILT_SI2E equ 4 

CENT_FILT_SHIFT equ 1 
CENT_FILT_SI2E equ 2 



inc_y 



/originally 3 was 2 
/originally 8 was 4 

/originally 2 was 1 
/originally 4 was 2 



bank ok: 



macro 

local bank_pk 
add si,Y_INC 
jnc bank_ok 

inc bank_number 
mov al f ACQUIRE_FLAG_ON 
add al,bank_number 
mov dx,LOW_CSRJPORT 
out dx,al 

endm 



set bank 



get_a_sum 



clamp_it : 
no__clamp: 



macro 

mov al,ACQUIRE_FLAG_ON 
add al , bank_number 
mov dx,LOW_CSR_PORT 
out dx,al 
endm 

macro 

local clamp_it 
local no_ciamp 
mov ax, [di] 
cmp word ptr[di+2], 0 
jnz clamp_it 

cmp ax,1000H 
jbe no clamp 



mov ax,1000H 



sub di,4 
endm 



ivg_seg 
here screen 



dw OAOOOh 
dw OBOOOH 



i_count_filt_sum dw 0 

l_count_filt_buffer dw COUNT FILT SIZE dup(O) 
l_count_filt_ptr dw 0 " 



l_x_filt_sum 



dw 0 
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l_x_filt_buffer aw CEOTMBSLT SIZE dup(O) 

l_x_filt_ptr dw 0 

l_y_filt_sum aw o 

l_y_filtj3iiffer dw CTNT_EILT_SIZE dup(O) 

l_y_filtj5tr dw 0 



r_count_f ilt_sum dw 0 

r_count_filt_buffer dw C0UNT_FIXTJSIZE dup CO) 

r_count_filt_ptr dw 0 

r__x_filt_sum dw 0 

r_x_filtjbuffer dw CENT_FILT_SIZE dup(O) 

r_x_filtjpT:r dw 0 

r_y_filt_sura dw 0 

-_v__filt_buffer dw CENT_FILTJ5IZE duo(O) 

r_y_filt_j>rr dw 0 



squares 



dw 3969, 


3844, 


3721, 


3600, 


3481, 


3364, 


3249 




dw 3136, 


3025, 


2916, 


2809, 


2704, 


2601, 


2500, 


2401 


dw 2304, 


2209, 


2116, 


2025, 


1936, 


1849, 


1764, 


1681 


dw 1600, 


1521, 


1444, 


1369, 


1296, 


1225, 


1156, 


1089 


dw 1024, 


961, 


900, 


841, 


784, 


729, 


676, 


625 


dw 576, 


529, 


484, 


441, 


400, 


361, 


324, 


289 


dw 256, 


225, 


196, 


169, 


144, 


121, 


100, 


81 


dw 64, 


49, 


36, 


25, 


16, 


9, 


4, 


j. 


dw 0 
















dw 1, 


4, 


a 


is. 


25, 


36, 


49, 


64 


dw 81, 


100, 


121,' 


144, 


169, 


196, 


225, 


256 


dw 289, 


324, 


361, 


400, 


441, 


484, 


529, 


576 


dw 625, 


676, 


729, 


784, 


841, 


900, 


961, 


1024 


dw 1089, 


1156, 


1225, 


1296, 


1369, 


1444, 


1521, 


1600 


dw 1681, 


1764, 


1849, 


1936, 


2025, 


2116, 


2209, 


2304 


dw 2401, 


2500, 


2601, 


2704, 


2809, 


2916, 


3025, 


3136 


dw 3249, 


3364, 


3481, 


3600,. 


3721, 


3844, 


3969 





corr_squares_j>t:r dw corr_squares_bufxer 
wav e_hisrory_ptz aw wave_hisrory_buffer 

.data? 

public _corr_sums_buf f er 
_corr_sums_fauffer dd CORRHISTORY_SIZE dup(?) 

corr_squares_buffer dw SQUAR£S_BUFFER_SIZE dupC?) 
wave_historyjDuffer dw CORR_HISTORY_SIZE dup(?) 

corr_count dw ? 

old_graph_0 dw 720*2 duD(?) 
old_ax_0 dw ? 

old_di_0 dw ? 

old_graph_l dw 720*2 dup(?) 
old ax 1 dw ? 
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old_di_l 

old_graph_2 

old_ax__2 

old_di_2 

old_graph_3 

old_ax_3 

old_di_3 

index 
pixel 

minjLndex_0 
min__index_l 
minjLndex_2 
min_index_3 

min_0 
min_l 
min_2 
aiin 3 



dw ? 

dw 720*2 dup(?) 
dw ? 
dw ? 

dw 720*2 dup(?) 
dw ? 
dw ? 

dw ? 

db ? 

dw ? 
dw ? 
dw ? 
dw ? 

dw ? 
dw ? 
dw ? 
dw ? 



old_period_jnarker dw ? 
period_marker_base dw ? 
old_period_pixel db ? 
periodjpixeijbase db ? 

left historyjbuffer db (X_COUNT * Y COUNT) dup (*>) 
right JiistoryjDuffer db (X_COUNT * YJCOUNT) dup (?) 

.code 



ban3c_number 
x_sum 

count 

zone count 



extrn eight: byte 

db 0 
dd ? 
dd ? 
dw ? 
dw ? 



int3 



int3 



public _int3 
proc near 

int 3 
ret 

endp 



_get_period 



public _get_period 
proc near 

push di 

mov di, CORR_SUMS__END 
mov dx, OFFFFH 



;Init mins 
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mov min_p,dx 
mov minJL,dx 
mov min_2,dx 
mov min_3,dx 

mov hx r 0 ;Init max 

mov CX r CORR_HISTORY_SIZE - 1 

Find First Minimum Sum 



risingr_loop_0 : 



nor_new_max_0 : 



found max 0: 



get_a_sum 
&np ax,bx 
jb not_new_max_0 

mov bx,ax 

loop rising__loop_0 

jmp find_best_min 



sub ax,bx 
neg ax 

cmp ax, BAND GAP 
jae found_max_0 

loop rising_loopjO 
jmp find_best_min 



loop falling_loop_0 
jmp find_besr_min 



ralling_loop_0 : 

get_a_sum 



not new min 0: 



found min 0: 



cmp ax,dx 

jae not_newjnin_J) 

mov dx,ax 

mov min_index_O r cx 

loop falling_loop_0 
jmp find_besr_min 

sub ax r dx 

cmp ax, BAND GAP 

jae f ound_minjD 

loop falling__loopjO 
jimp f indjDest_min 

mov minj) r dx 

mov dx, OFFFFH ;Init mins 

IDav *«r0 ; init max 
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loop rising_loop_i 
jmp findjDestjnin 



Find Second Minimum Sum 



rising_loop_l : 



not new max 1: 



found max 1; 



get_a_sum 
amp ax,bx 
jb not_newjnax_l 

mov bx,ax 

loop rising_loop_l 

jmp f indjbest_xnin 



sub ax,bx 
neg ax 

cmp ax r BAND GAP 

jae found_max_l 

loop rising_loop_l 
jmp find_best_min 



loop falling_loop_l 
jmp findjbest_min 



falling_loop 1: 



get_a_sum 
cmp ax,dx 
jae not_new_min_l 

mov dx, ax 

mov min_index_l,cx 

loop failing__loop_l 
jmp f indjbest_min 

sub ax,dx 

cmp ax,BANDGAP 

jae findJbesiDjnin 

loop falling_loop__l 
jmp f indjbest_min 



Decide which minimum sum is besr 



find best min: 



mov ax„min_0 
cmp ax,dx 

ja second_is_smallest 
jz both_mins_equal 
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First one is smallest 
;*************************«*********^^ 

f irst_is_best : 

cmp ax r MAX_GO0DjCORR 
jb first^is^ok 
jrnp nojgoodjcorr 

first_is_pJc: 

mov cx,min_index_0 
mov ax r C0PR_HIST0RY_SI2E 
sub ax,cx 
jmp end_fms 



Second one is smallest 

second__is_smallest: 

sub ax,dx ; First - Second 

cmp ax, DECXSIONJZONE 
jle first_is_best 

second_is_best: 

cmp dx,MAX_GOOD_CORR 
jb second_isjo)c 
jmp no_good_corr 

second__is_ok: 

mov cx,min_index_l 
mov ax,CORRJHISTORY_SIZE 
sub ax r cx 
jmp end_fms 

both _mins_jequal : 

crop ax, OFFFFH 
jnz 5irst_is_best 

no_good_corr :. 

mov ax,0 



end fms: 



pop di 
ret 



__get_period endp 
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campute_correlation (new_data) 

4 

SI ptr to sums 

DI ptr to squares 

BX for looJcup 

BP ptr to wave history 

CX counter for history size 



public compute correlation 
_compute_correlation proc nelr " 

push bp 
mov bp, sp 
push si 
push di 



not w h el: 



mov ax, [bp+4] 

lea si,_corr_surasjbuffer 
mov di , corr_squares__ptr 
mov bp,wave_history_ptr 

mov cx, CGRR_HISTORY_SIZE 
add bp, 2 

cmp bp,WAVE_HISTORY END 
jnz not_w_h_el 
lea bp,wave_history_Jbuffer 

mov [bp], ax 



;Get new data point 
; Setup pointers 

/Initialize counter 
;Word increment history pointer 

/Store new data in history buffer 



Subtract current from historic data 

Square the result 

Subrtact old square from sum 

Add new square to sum 

Put new sum in buffer 



comput e_corr_loop : 

mov bx, [bp] 
sub bx,ax 
shl bx,l 

mov dx, squares [bx] 

mov bx, [di] 
sub [si],bx 
sbb word ptr[si+2],0 

mov [di],dx 

add [si],dx 

adc word ptr[si+2J,0 



;Get historic data 

;Get difference between historic and new 

;Find square by lookup 

/Get previous square 
/Remove it from long sum 



/Store new square in squares buffer 
/Add new square to sum 
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; Adjust pointers and handle wraparound 

add si r 4 ;Long increment sunt pointer 

add di,2 ;Wbrd incr e ment squares pointer 

add bp,2 ;Word increment wave history pointer 

cngj di,roi»_SQtZM^_END 
jnz not_c_s_e 

lea di, eorrjsquares Jbuffer 

notjcjsja: 

cmp bp,WAVE_HISTORy_END 

jnz not_wh_e2 

lea bp, wave history buffer 

not_w_h_e2 : 

loop compute_corr_loop 

Save pointers for later 
;*******************•******************* 

mov corr_jsquares_ptr, di 
mov wave_historyjptr, bp 

pop di 
pop si 
pop bp 
ret 

_comput:e_correlation endp 




display_corr_graphs (new_data r Deriod) 

4 6 



public _display_corr_graphs 
jiisplay_corr_graphs proc near 

push bp 
mov bp, sp 
push si 
push di 

mov es,hercjscreen 

mov bx, index 
mov dl,pixel 



Add current data to graph 0 
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blank down Oc: 



blank_up_Oc : 



mov di,old_graph_0[bx] 
mov oc,old_graphJ)[bx+2] 

crop di, 0 

jz do_points_0c 

crop cx, 0 

jl blank_up_Oc 

jg blank_down_Oc 

xor es: [di] , dl 
jmp dojpoints_Oc 



call vert_line_d 
jmp do_points_Oc 



neg cx 

call vert line u 



do_points__0c: 



in_range_Oc: 



line_down__Oc : 



line_up_0c: 



next_pixel_Oc: 



mov ax r [bp+4] 
crop ax, 64 
jbe in_range_Oc 
mov ax, 64 



mov cx, old_ax_0 
mov di,old_di__0 
sub cx, ax 

mov old_graph_0 [bx] , di 
mov old_graphJD [bx+2 ] , cx 
mov old_ax_JD,ax 

cmp cx, 0 

jl line_up_0c 

jg line_down_Oc 

xor es : [di ] , dl 
jmp next_pixel_Oc 

call vert_line_d 
jmp next_jpixel_Oc 



neg cx 

call vert line u 



;Get di of old graph 
;And count and direction 



;Is old line up, down or horizontal 



/Horizontal (blank one dot) 



;Get count and direction 

; Stash DI 

;And CX in array 

; Stash AX 

;Is line up, down or horizontal? 



; Horizontal (one dot) 



mov old_di_0,di 

ror pixel, 1 
jnc no_jLnc_c 
inc old di 0 



; Stash DI 

; Shift bit pattern 
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no_jLnc_c: 

add index, 4 
cmp index, 720*4 
jl graph_done_c 

mov index, 0 
sub oldj±L_0,90 

graph_done_c : 



_ Display entire correlation on graph 1 

T *****************************^**********^ 

mov si,CORR_SUMS__END 
mov old_di_l, 90*42 
mov old_ax_l,0 

mov corr_count, CORRJilSTORY SIZE 
mov dl,10000000B 
mov bx,0 

corr_graph_loop: 

mov di , old_graph_l [bx ] ;Get di of old graph 
mov cx, oldjgraph_l [bx+2] ;And count and direction 

cmp di,0 

jz do_points_JLc 

cx,0 ;i s old line UD,down or horizontal 

jl blanJc_up_lc 
jg blanJc_down_lc 

xor es:[di],dl ; Horizontal (blank one dot ) 

jmp do__points_lc 

blank_jdown_lc: 

call vert_ line_d 
jmp do_j>oints_lc 

blanx_up__lc: 

neg cx 

call vert_line_u 

GO_j>oints_lc: 

mov ax, [si] 

mov cl,6 
shr ax, cl 

mov cx, [si+2] 
sub si, 4 

cmp cx, 0 

jz no__uw 

mov ax, 64 

jmp injrange_lc 
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no uw: 



injrangeJLc: 



line down lc: 



linejup_lc: 



next_Dixel_lc: 



no inc lc: 



cmp ax, 64 

jbe in_range_lc 

mov ax, 64 



mov cx,old_ax_l 
mov di,old_di_l 
sub cx,ax 

mov old_graph_l [bx] , di 
mov old_graph_l [bx+2 ] ,cx 
mov old_ax__l,ax 
mov old_di_l,di 

cmp cx,0 

jl line_up_lc 

jg line_down_lc 

xor es: [di],dl 
jmp next _j3ixel_lc 

call vert_line_d 
jmp next_pixel_lc 



neg cx 

call vert line u 



dec_count_lc: 



graph_done_JL c : 



mov old_di_l,di 

ror dl,l 

jnc nq_inc_lc 

inc old di 1 



add bx, 4 

cmp bx, 720*4 
jl dec_count_lc 

mov bx,0 

sub old di 1, 90 



dec corr_count 
jz graph_done_le 
jmp corr_graph_loop 



/Get count and direction 

; Stash DI 
;And CX in array 
; Stash AX 
; Stash DI 

;Is line up, down or horizontal? 



/Horizontal (one dot) 



/Stash DI 

/Shift bit pattern 



Now display detected period marker 



mov di,oldj?eriod_marJcer 



1990 Walt Disney Imagineering 



SUBSTITUTE SHEET 



WO 93/22762 



PCT/US93/03667 



- 112 - 



mov dl r oldj>eriodj)ixel 

mov cx,10 

call vertlinejd 

mov ax, [bp+6] 
div eight 

mov cl,ah 

mov dl„10000000B 

ror dl,cl 

mov old_period_pixel,dl 
mov ah,0 

mov di,period_markerJbase 
add di,ax 

mov oId_period_marker , di 

mov cx,10 

call vert_line_d 

pop di 
pop si 
pop bp 
ret 

_dispiay_corr_graphs enqp 

public _align_camera 
_align_camera proc near 

push di 

mov es, ivg__seg 



ac Iood: 



ac _ly_J-Oop: 



ac_lx_loop: 



mov bank_number, INITIAL BANK 
setbanfc 

mov di , UFPER_LEFT 
mov bx r Y COUNT 



push di 

mov cx r X_COUNT 
mov al, OFFH 



mov byte ptr es:[di] r 0FFH 
add di,X_INC 
loop ac_lx_loop 

pop di 

add di # yjrNC 
jnc lbank_pk 

inc bank number 
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mov al r ACQUIRE_FLAGjON 
add al , bank_number 
mov dx , LOW_CSRJ>ORT 
out dx,al 

lbank_pk: 

dec bx 

jnz ac_ly_loop 



mov bank_number, INITIALJBANK 
set_bank 

mov di , UPPER_RIGHT 
mov bx,Y_COUNT 

ac_ry_loop: 

push di 

mov cx,X_COUNT 
mov al, OFFH 

ac_rx_loop: 

mov byte ptr es:(di],0FFH 
sub di,X_INC 
loop ac_rx_loop 

pop di 

add di,Y_INC 
jnc rbank_ok 

inc bank__number 
mov al , ACQUIRE_FLAG_ON 
add al , bank_number 
mov dx^LOW^CSRJPORT 
out dx,al 

rbankjDk: 

dec bx 

jnz ac_ry_loop 

mov ah,l 
int 16H 
jnz ac_end 
jmp acJLoop 

ac_end: 

mov ah, 0 
int 16H 

pop di 
ret 

_align_camera endp 



init_historyJbuf f er ( ) ; 
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public _initJiistoryjDuffers 
J.nit_Mstory_buf f ers proc near 

push ds 
push si 
push di 

push ds 
pap es 

lea di f lef t_historyjDuf f er 

mov ds, ivg_seg 
xnov si,UP£ER_LEFT 
mov bx,X COUNT 



ih_lx_loop: 



ihJLy_Ioop: 



mov banJc_number, INITIAL BANK 

set_banJc 

mov ra,r_COUNT 

push si 



mov al, [si] 

stosb 

inc_j\ 

loop ih_ly_loop 



pop si 

add si,x_INC 
dec bx 

jnz ih_lx_loop 



ih_rx_loop: 



ih_ry_loop: 



lea di,right_Mstory_buffer 

mov sirUPPERHIGHT 
mov bx,X COUNT 



mov bank_number , INITIAL BANK 
setJbanJc ~~ 
mov cx r Y_COUNT 
push si 



mov al r [si] 

stosb 

inc_y 

loop ih_ry_loop 
pop si 

sub si r X_INC 
dec bx 

jnz ih_rxJLoop 
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pop di 
pop si 
pop ds 
ret 

_init_history_bi2ffers endp 



* int get_l_centroid(int *); 

* filtered_count = get_l_centroid(&filtered_x, ^filtered v) ; 



public _get_l_centroid 
_? et _l_centroid proc near 

push bp 
mov bp,sp 
push si 
push di 

push ds 

push ds 
pop es 



Find first qualifying pixel 

And start filling history buffer 

lea di, left_history_buffer 
mov as, ivg_seg 
mov si, UPPER_L£FT 

mov word ptr x_sum,0 
mov word ptr x_sum[2], 0 
mov word ptr y_sum, 0 
mov word ptr y_sum[2],0 
mov count, 0 

mov bx,X COUNT 



mov bank_number, INITIAL BANK 
set_bank ~~ 
mov cx,Y_COUNT 
push si 



mov al, [si] 
mov ah,al 
sub ah,es: [di] 
stosb 
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cap ah,I£W_THKESEQLD 
jb notj?ualJL_fp 
cap ah f fiXGHJTHHESHQLD 
ja notjqual_l__fp 

mov zone_count,ZONE_SIZE 



not _quai_l_fp : 

mav bvte ptr[si] r 080H light scan rectangle dimly 

inc_y 

loop rp_l_y_jLoop 

pop si 

add si,X_INC 

dec bx 

jnz fp_ljcJLoop 

mov count, 0 

jmp compute_l__centroid 

Get centroid of qualifying pixels 



gc_i _x_ioop: 



ge_I_v_loop: 



mov banJc_number, INITIAL BANK 
set_banJc "~ 
mov cx,Y_COUNT 
push si 



mov al, [si I 
mov ah,al 
sub ah, es : f dij 
stosb 

anp ah, LOWJTHRESHOLD 
jb not_qual_i_gc 
cmp ah, HIGH_THRESHOLD 
ja not_qual__I_gc 



add_to 1 sums: 



mov byte ptr[si],OFFH 

add word ptr x_sum,bx 
adc word ptr x_sumf2],0 

add word ptr y_sum, ex 
adc word ptr y_sumf2],0 



inc count 

not jgualJLjgc : 



inc_y 
loop gc_l_y_loop 



1990 Walt OisMy Ia»ginB«ring 



SUBSTITUTE SHEET 



WO 93/22762 



PCT/US93/03667 



- 117 - 



pop si 

add si,X_lNC 
dec bx 

jz conrpute^l^centroid 

dec zone_count 
jnz gc_l_x_loop 



'^♦w^*******^ 1113 * 1 fill ing history buffer 
fh__l_x_loop: 

mov bank_number, INITIAL BANK 
set_ban3c ~~ 
mov cx,Y_COUNT 
push si 

fh_l_y_loop: 

mov al, [si] 

stosb 

inc_y 

loop fh_I_y_loop 
pop si 

add si,X_INC 
dec bx 

jnz fh_l_x_looD 



; Compute the centrcid if possible 

; -***»********»^^5**"*»^ using moving wmaow averacmg 

compute_l_centroid: 

pop ds 

shr count , 1 ;+++ 

cmp count ,MIN_CCUNT 
jbe no_l centroid 



Compute and filter x centroid 



mov ax, word ptr x_suxn 
mov dx,word ptr x_sum[2] 
div count 

mov si, l_x_f ilt_ptr 

mov bx, l_x_f ilt_sum 

sub bx,l_x_filtjbuffer[sil 

mov l_x_filt_buffer(si],ax 

add ax,bx 
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mov l_x_filt_sum,ax 
mov cl,CEOT_FILrj5HXFT 
shr ax, cl 

add si, 2 

onp si,CENT_FH«T_SI2E*2 
jnz ljK_filt_ok 
mov si,0 

mov i_x_f ilt jptr, si 

mov di, fcp+4] 
mov [di],ax 



Compute and filter y centroid 



i_v_filtjoJc: 



mov ax, word ptr y_sum 
mov dx,word ptr y_sum[2] 
div count 

mov si,l_y_filtj>tr 

mov bx, l_y_£ ilt_sum 

sub bx,l_y_filt_buffer[si] 

mov l_y_filt w buffertsi] ,ax 

add ax,bx 

mov l^_filt_sum,ax 
mov cl , CEOT_FILT SHIFT 
shr ax,cl 

add si r 2 

cmp si r CENT_FH.TjSI2E*2 
jnz l_y_filt_ok 
mov si,0 

mov l_y_£ ilt_ptr, si 

mov di, [bp+6] 
mov [di],ax 



• *»******-*,**** F ii 1:er count using moving window averaging 
no_i_centroid : 



mov si, l_count_f iltjptr 

mov ax r l_count_filt_sum 

sub ax, l_count_f ilt_buf fer [si] 

mov bx, count 

mov l_countjfiltj5uffer[siJ,bx 
add ax,bx 

mov ljDoimtJC ilt_sum r ax 
mov cl,COTNT_FILT_SHIFT 
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shr ax,cl 
add si, 2 

cop si, C0UNT_FILTJSIZE*2 
jnz l_count_filt_ok 
mov si,0 
l_count_filt_ok: 

mov l_count_f ilt_ptr, si 

pop di 
pop si 
pop bp 
ret 

j?et_l_centroid endp 



int get_r_centroid (int *); 
filtered_count - get_r_centroid(&filtered_x, &£ iltered_y) ; 



public _get r centroid 
_get_r_centroid proc near 

push bp 
mov bp,sp 
push si 
push di 

push ds 

push ds 
pop es 



Find first qualifying pixel 

And start filling history buffer 



lea di,right_historyjbuffer 

mov ds,iva_seg 
mov si,UPFER_RJGHT 

mov word ptr x_sum, 0 
mov word ptr x_sum[2],0 
mov word ptr y_sum, 0 
mov wopd ptr y_sum[2] , 0 
mov count, 0 

mov bx,X COUNT 



fp_r_x_loop: 
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mov banfc_number , initial bank 
setbanfc ^ 
mov CX,Y_COUNT 
push si 



mov al,Isi] 
mov ah,al 
sub ah,es: [di} 
stosb 

anp ahrLOWJTHRESHOLD 
jb notjqual_r_fp 
anp ah, HIGH_THF£SHOLD 
ja not_<jual_r_fp 

mov zone_count,ZONE_SIZE 
jmp add_to_jr_sums 



not_qualjr__fp: 



mov byte ptr[si],080H light grid dimly 

inc_y 

loop fe_rjr_loop 

pop si 

sub si,X_INC 

dec bx 

jnz fp_rjc_loqp 

mov count, 0 

jmp ccsnpute_r_centroid 



" Get centroid of qualifying pixels 



gc_r_x_ioop: 



? c -. r _vL.ioop: 



mov banic_number, INITIAL BANK 
set_banJc ~" 
mov cx,Y_COUNT 
push si 



mov al, [si] 
mov ah,al 
sub ah,es: fdij 
stosb 

anp ah,LOWJTHRESHOLD 
jb not_qual_r_gc 
cmp ah,HIGH_THHESHOLD 
ja not_qualjr_gc 



add_t.c__r_sums : 



mov byte ptrfsi] , OFFH 
add word ptr x sum,bx 
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adc word ptr x_sum[2],0 

add word ptr y_suxa r cx 
adc word ptr y_sum[2],0 



inc count 

not jqual_r_gc : 



inc_y 

loop gc__r_y_loop 
pop si 

sub si,X_INC 
dec bx 

jz compute_r_centroid 

dec zone_count 
jnz gc_r_x_loop 



Finish filling history buffer 



-_r_:-:_loop: 



fh_r_j/__l oop : 



mov bank_number, INITIAL BANK 

set_ban)c 

mov cx,Y_COUNT 

push si 



mov al, [si] 

stosb 

inc_y 

loop fh r v Iood 



pop si 

sub si,X__INC 
dec bx 

jnz fh r ;< Iood 



Compute the centroid if possible 
*^L filter iZ USrng movin <? window averaging 



compute_r_centroid: 

pop ds 

shr count,! 



crap count, MINJTOUNT 
jbe no r cenrroid 



Corapute and filter x centroid 
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mov ax, word ptr x_sum 
mov dx,word ptr acjsxaa[2] 
div count 

mov si,r_x_filt_ptr 

mov bx,r_x_fiit_stni 

sub bx, r_x_f iltjsuf f er f si J 

mov r_x_f ilt_buf f er [si ] , ax 

add axrbx 

mov rjc_filt_sum r ax 
mov ci,CENT_FILT_SHIFT 
shr ax,cl 



r_x_filt_pk: 



add si, 2 

cmp si,C3Nr_FILT_j3IZE*2 
jnz r_x_£ilt_ofc 
mov si,0 

mov r_x_f ilt_ptr, S i 

mov di, [bp+4] 
mov [dij ,ax 



r_y_filt_o)c: 



Compute and filter y centroid 



mov ax, word ptr y_sum 
mov dx,word ptr y_sum[2] 
div count 

mov si r rj/_filt_prr 

mov bx r rj/_5iit_sum 

sub bx,r_v_filtjDuffer[si] 

mov r_y_filtjbuffer [si] r ax 

add ax,bx 

mov r _y_fi!t_sum,ax 
mov ci,C2NT_FILT_SHIFT 
shr ax r ci 

add si, 2 

cmp. si,CENT_FILTJ5IZE*2 
jnz r_y_filt_oJc 
mov si,0 

mov r_y_filt_ptr , si 

mov di, [bD+6] 
mov [di],ax 

Filter count using moving window averagxng 
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no_r_centroid: 

mov si, r_count_filt_ptr 

mov ax, r_count_f ilt_sum 

sub ax,r_count_filt_buffer[si] 

mov bx,count 

mov r_count_filt_buffer(si] ,bx 
add ax,bx 

mov r_count_f ilt_sum, ax 
mov cl, COUNT_FILT_SHIFT 
shr ax,cl 

add si, 2 

crap si,COUNT_FILT__SI2E*-2 
jnz r_count_filt_ok 
mov si,0 
r_count_filt_ok: 

mov r_count_f ilt_ptr, si 

pop di 
pop si 
pop bp 
ret 

_ger_r_centroid endp 

public _wait for odd field 
_wait_f or_odd_field proc near " " 



wfef_loop: 



wfof Iood: 



mov dx,HIGH_CSRJPORT 
in al,dx 

test al,10000000B 
jnz wfef_loop 



in al,dx 

test al,10000000B 
jz wfof_loop 



;loop while field is odd 4 



;loop while field is even 



ret 

_wait_f or_odd_f ield endp 



public _init sparkle lut 
init_sparkle_lut proc near" 



?Set OLUT 0, Set ILOT o. No Write Protect 



1990 Walt Disney ImaginMring 



SUBSTITUTE SHEET 



WO 93/22762 



PCT/US93/03667 



- 124 - 



slutJLoop: 



mov ai r 0OO00OOOB 
mov dx,HIGH_CSR PORT 
out daedal 



mov ax, 0 

mov dx , LtJT_ADDRESS PORT 
out dx,al ~ 
mov dx,INPOT_IOT DATA PORT 
out dx,al ~ " 

xchg ah,al 

mov dx,REDJLUTJ)ATA PORT 
out dx,al ~~ 
mov dx, GREEN_LUT DATA PORT 
out dx,al ~~ 
mov dx,BLUE_LOT DATA PORT 
out dx,al ■ 

xchg ah,al 

inc al 

jnr slur Iood 



mov dx, LUT_ADDRESS PORT 
mov al, OFFH 
out dx,al 

mov dx, RED_LOT DATA PORT 
out dx,al ~ ~ 
mov LOT DATA r>ORT 

out dx,al ~ ~" ~ 
mov dx,BLUE_LUT DATA PORT 
out dx,al ~ ~ 

dec al 

mov dx, INPUT LOT DATA PORT 
out dx,al " ~ " 



mov al,ACQUIRE_FLAG ON 
mov dx,LOW_CSR PORT"" 
out dx,al ~ 



ret 

J.nir_sparJde_lut endp 



P^ 110 -i*** norm lut 
J.nit_noxm_iut proc near ~~ ~ 



;Set OLOT 0. set ILUT 0, No Write Protect 
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raov al,00000000B 
mov dx,HIGH_CSR_PORT 
out dx,al 

mov al,0 

mov dx 9 LUT_ADDR£SS_PORT 
out dx,al 

mov oXRED_LUT_DATA_PORT 
out dx,al 

mov dx , GREEN_LUT_D ATA PORT 
out dx,al "~ 
mov dx, BLUE_LUT_DATA PORT 
out dx,al ~ 
mov dx, INPUT_LUT_DATA PORT 
out dx,al 

inc al 

jnz nlut_loop 

mov al , ACQUIKE_FLAG ON 
mov dx , LOW_CSRJPORT 
out dx,al 



ret 

J-nit_norm_lut endp 



public _init_corr aravhs 
.ni w _corr_graphs proc near " 

push di 

push ds 
pop es 

mov index, 0 

mov pixel, 10000000B 

mov ax, 0 

mov old_di_0, 90*21 
mov old_ax_0, 0 

lea di,old_graph_0 
mov cx, 720*2 
rep stosw 

mov old__di_l, 90*42 
mov old_ax_l,0 

lea di, old_graph_l 
mov cx, 720*2 
rep stosw 
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lea di r _corr_sums_buffer 
mov cx r C0RR_HIST0RY_SI2E *2 
rep stosw 

lea di, corx^squares_buf fer 
mov cx r SQUM?ES_BUFFER_SI2E 
rep stosw 

lea c^rwave^MstoryJbxaffer 
mov cx,O)I«_HIST0HyjSIZE 
rep stosw 

mov old_period_pixel,10Q00000B 

mov old^jjeriodjnarker, 90*42 
mov periodjnarJcerjDase, 90*42 

pop di 
ret 

_ - r *"_corr_graphs endp 

;™:™;™^^ — ****^ 

_ public _init_f our_graphs 

_-n:.t_rour_graphs .prqc near 

push, di 

push ds 
pop es 

mov index, 0 

mov pixel, 10000000B 

mov ax,0 

mov oldjdi_O r 90*21 
mov old_ax_0, 0 

lea di,old_araDh_0 
mov cx, 720*2 
rep stosw 

mov old_di_l, 90*42 
mov old_axJL,0 

lea di,old_graph_l 
mov cx, 720*2 
rep stosw 

mov old_di_2, 90*63 
mov old_ax_2, 0 
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lea di,old_graph_2 
mov cx, 720*2 
rep stosw 

mov old_di_3, 90*84 
mov oldjax_3,0 

lea di,old_graphJ3 
mov cx, 720*2 
rep stosw 

pop di 
ret 



- ini t_f our^graphs endp 



display_four_graphs(yO, yl, y 2, y3) where y»s are from 0-64 

4 6 8 10 



, public _display four graphs 

J2isplay_four_graphs proc near - ~~ 



push bp 
mov bp,sp 
push di 

mov es, herc_screen 

mov bx, index 
mov ril, pixel 



bianic_graph_0: 



blank_down_0 : 
blanJc_up_0 : 



mov di,oldj?raph_0[bx] ;Get di of old oraph 
mov cx,old_graph_0fbx+2) ;And count and direction 

cmp di, 0 

jz do_points_0 

j?blMk up 0 ' IS ° ld line up ' down or ^izontal 

jg blank~down_0 

xor es: [di] ,dl /Horizontal (blank one dot) 

jmp dq_points_0 

call vert_line_d 
jmp dojpoints_0 

neg cx 

call vert line u 
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do_pointsjO: 



in_range_0: 



line_down_0: 



line ud 0: 



mov ax, [bp+4] 
asp ax, 64 
jbe iirrangejD 
mov ax, 64 



mov cx,oldjaxJ) 
mov di,oldjciiJ) 
sub cx,ax 



;Get count, and direction 



mov old_graph_0 [bx] , di ; Stash DI 
mov old_graphJ)rbx+2],cx ;And CX in array 



mov old_ax_p,ax 

anp cx, 0 

Jl line_up_0 

jg line_down_0 

xor es : [di] , dl 
jmp blanic_graph_l 



call vert_iine_d 
jmp blank_graph_l 



neg cx 

call vert_line_u 



; Stash AX 

;Is. line up, down or horizontal? 

/ Horizontal Cone dot) 



blanjc__graph_l : 



mov old_di_0,di 



; Stash DI 



mov di r oldLgraphl[bx) -Get di of old oraoh 
mov cx,old_graph_irbx+2] ;And count and direction 

cap di, 0 

jz do_points_l 



blank_down_l; 

blank_up_X: 

dq_j5ointsj.; 



cmp cx,0 

Jl blanJcjipjL 

jg blank_down_l 

xor es: fdi],dl 
jmp dqjpoints_l 

call vert_line_d 
jmp ctojpoints^l 



neg cx 

call vert line u 



;Ts old line up, down or horizontal 



/Horizontal (blank one dot) 
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in_range_l: 



line down 1: 



line ud l: 



mov ax, [bp+6] 
cap ax, 64 
jbe in_range_l 
raov ax, 64 



mov cx,old_ax_l 
mov di,old_di_l 
sub cx,ax 



;Get count and direction 



mov old_graph_l flbx ) , di ; Stash DI 

mov old_graph_irbx+2],cx ;And CX in array 

mov old_ax_l,ax ; Stash AX 

mov old_di_l,di ; Stash DI 



r cx,0 
jl line_up_l 
jg line_down_l 

xor es: [di],dl 
jmp blank_graph_2 



call vert_line_d 
jmp blank_graph_2 



neg cx 

call vert line u 



rls line up, down or horizontal? 



; Horizontal (one dot) 



bl anagraph 2 : 



b 1 an)c_down_2 : 



blank ud 2: 



do_points_2: 



mov old_di 1, di 



; Stash DI 



mov di,old_graph_2fbx] ;Get di of old craoh 
mov cx,old_graph_2 [bx+2] ;A nd count and direction 

cap di, 0 

jz do_points_2 



cmp cx.,0 

jl blanJc_up_2 

jg blan)c_down_2 

xor es: [di],dl 
jmp do_points_2 



call vert_line_d 
jap do_points_2 



neg cx 

call vert line u 



mov ax, [bp+8] 



;Is old line up, down cr horizontal 



;Horizontal(blanJc one dot) 
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inxange_2: 



.Line down 2: 



line ud 2: 



cmp ax, 64 

jbe in_range__2 

mov ax, 64 



mov cx,old_ax_2 
mov d&,o!ddi_2 
sub cx f ax 



/Get count and direction 



mov old_graph_2 [bx] r di ; Stash DI 
mov old_graph_2[bx+2] , cx ;And CX in array 
mov old_ax_2,ax 
mov old_di_2,di 



crop cx, 0 

jl line_up_2 

jg line_down_2 

xor es: [di] ,dl 
jmp blanx_graph_3 



call vert_line_d 
jmp blanJcjgraph_3 



neg cx 

call vert line u 



/Stash AX 
; Stash DI 

/Is line up, down or horizontal? 



r- Horizontal (one dot) 



bianic_graph_3: 



blank_down_3 : 



blanJc_up_3 : 



dq_points__3 : 



mov old_j±L_2 r di 



; Stash DI 



mov di,old_graph_3 [bx] /Get di of old orauh 
mov cx,oId_graph_3[bx+2] /And count and direction 

asp di r 0 

jz dq_points_3 



cmp cx,Q 

jl blankjap_3 

jg blanJc_down_3 

xor es: [di] ,dl 
jmp do_points_3 

call vert_line_d 
jmp do_points_3 



neg cx 

call vert line u 



mov ax, [bp+10] 
cmp ax,64~ 



/Is old line up, down or horizontal 
/Horizontal (blank one dot) 
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in_range_3 : 



jbe in__range_3 
mov ax, 64 



line_down 3: 



line up 3: 



next_pixel: 



no inc: 



graph_done : 



mov GXipOLdjK 3 
mov di,old_di_3 
sub cx,ax 

mov old_graph_3[bx],di 
mov old_graph_3 [bx+2 ] , cx 
mov old_ax_3,ax 
mov old_di_3,di 

cap cx, 0 

jl linejup_3 

jg line_down_3 

xor es: [di],dl 
jmp next_pixel 



call vert_line_d 
jmp next oixel 



neg cx 

call vert line 



mov oldj±L_3,di 

ror pixel,! 
jnc nq_inc 
inc oid_di_0 
inc old_di_l 
inc old_di_2 
inc old di 3 



add index, 4 
crop index, 720*4 
jl graph_done 

mov index, 0 
sub old_di_0, 90 
sub old_di_l,90 
sub old_di_2,90 
sub old di 3,90 



pop di 
pop bp 
ret 



;Get count and direction 

; Stash DI 
;And CX in array 
; Stash AX 
; Stash DI 

;ls line up, down or horizontal? 



.•Horizontal (one dot) 



; Stash DI 

; Shift bit pattern 



_cusplay_four_graphs endp 
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Draw a vertical line CONN from dotpos di for length cx 
°sing pattern in dl 

vert_lineji proc near 

cm© di,6000H 
jge line3 

cmp di,4000H 
jge line2 



lineO: 



linel: 



iine2; 



Iine3 : 



cmp di,2000H 
jge linel 

xor es: [di], dl 
add di, 2000H 
locp linel 
ret 

xor es: fdi], dl 
add di, 2000H 
loop line2 
ret: 

xor es:[di] r dl 
add di, 2000H 
loop line3 
ret 

xor esrfdi], dl 
sub di, 5fa6H 
loop lineO 
ret 

vert_line_d endp 

^— — ~« 

vert_line_u proc near 

cmp di,2000H 
jl lineO_u 

cmp di,4000E 
jl linelju 



line3 u; 



cmp di,6000H 
jl line2_u 



xor es:fdi], dl 
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line2 u: 



linel u: 



lineO u: 



vert_line_u 



sub di, 2000H 
loop line2_u 
ret 

xor es:(di], dl 
sub di, 2000H 
loop linel u 
ret 

xor es: [di] , dl 
sub di, 2000H 
loop lineO_u 
ret 

xor es: [di] , dl 
add di, 5fa6H 
loop line3_jj 
ret 

endp 

end 



x_get_period 



public x_get_period 
proc near 

push di 

mov di,CORR_SUMS_END 

mov dx, OFFFFH 
mov min_0,dx 
mov min_l,dx 
mov min_2,dx 
mov min 3,dx 



mov b *'° ;lnit max 

mov CX,CORR_HISTORY_SIZE - 1 



;Init mins 



Find First Minimum Sum 



rising_loop 0: 



not_nev/jnax__0 : 



get_a_sum 
anp ax,bx 
jb not_new_max_0 

mov bx,ax 

loop rising_loop_0 

jmp find best min 



sub ax,bx 
neg ax 

cmp ax f BANDGAP 



<D 
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found max 0: 



jae founctaaxJ3 

loop rising_loop_0 
jmp findjbesrjnin 

loop falling_loop_0 
jmp findjoestjnin 



falling_loop_0 : 

get_a_sum 



cmp ax, dx 
jae not_new_min_0 

mov dx,ax 

mov min_indax_0,cx 

loop falling_loop_0 
jmp f ind_best:_rain 



not_newjnin_0 : 



found min 0: 



sub ax,dx 

cmp ax, BAND GAP 

jae found_min_0 

loop falling_loop_0 
jmp find_best_mxn 

mov min_0,dx 

mov dx, OFFFFH ; I nit mins 

mov bx,0 ;Init ^ 

loop rising_loop 1 
Imp find_besr_mxn 



Find Second Minimum Sum 



rz.sing_Loop_l : 

get a sum 



iot_new_max_l : 



cmp ax,bx 

jb not_newjnax_l 

mov bx,ax 

loop rising_loop__l 

jmp find_best_min 

sub ax,bx 
neg ax 

cmp ax, HAND GAP 
jae found_maxJL 

loop rising_loop_l 
imp f ind best min 



° 1990 Walt Di«n«y Imagin««ring 



SUBSTITUTE SHEET 



WO 93/22762 



PCT/US93/03667 



- 135 - 



foundjnax_l: 

loop f alling_loop_l 
jnp fiiidbestjnin 

f alling_loop_l : 

anp ax,dx 

jae not_nevjnin_l 

mov dx,ax 

mov rain_index_l, cx 

loop falling_loop_l 
jinp findjbestjnin 



not new min 1: 



found min 1: 



sub ax, ax 

crap ax, BAND GAP 

jae foundjnin_l 

loop falling_loop_l 
jinp findjbest_min 



mov min_l,dx 

mov dx, OFFFFH ; mit mins 

mov bx,0 ;Init ^ 

loop rising_loop_2 
jmp find_best._min 



Find Third Minimum Sum 



rising_loop_2 : 



not_new_max_2 : 



foundjnax_2: 
f alling_loop_2 ; 



get_a_sum 
cmp ax,bx 
jb not_newjnax_2 

mov bx,ax 

loop rising^_loop_2 

jmp find_best:_min 



sub ax,bx 
neg ax 

amp ax, BAND GAP 
jae fourtd_max_2 

loop rising_loop_2 
jmp f indjbestjnin 

loop falling_loop_2 
jmp f ind_bestjain 
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cap ax,dx 

jae not:_new_inin_2 

mov dx„ax 

mov m±n_index_2 , cx 

loop falling_ioop_2 
jmp fizid_best_jnin 



not_newjnin_2 : 

sub ax r dx 



found rain 2: 



cmp ax, BANDGAP 
jae found_jnin_2 

loop falling_loop_2 
jmp findjbesrjnin 

mov min_2 r dx 

raov frOEEETH ;Init ^ 

no,r bx '° ;lnit max 

loop risingr_loop_3 
jmp find_best_min 



Find Fourth Minimum Sum 

rising_loop 3; 

getja_sum 
cmp ax,bx 
jb not_newjnax_3 



********************** 



notjnew_max_3 : 



mov bx,ax 

loop rxsing_loop_3 

jnp f ind_besc_jain 

sub ax„bx 
neg ax 

™p ax, BAND GAP 
jae fbundjnax_3 

loop rising_loop_3 
jmp findjbest_min 

loop falling_loop_3 
jnp findjbest_min 

9®tja_sum 
cmp ax,dx 
jae not_newjnin_3 

mov dx,ax 
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jle first is best 



second_isJbest : 



secondJ.s_ok: 



mov cx , min_index_l 
cap CX,MAX_GOOD_COKR 
jb secondjLsjok 
jmp no_good_corr 

mov ax,CORR_HISTORi: SIZE 
sub ax,cx ~~ 
jmp end_£ms 



second_not_smallest : 

cmp cx,dx 

ja third_not_smallest 
jz fourth_not_smallest 



Third one is smallest 



sub ax, cx 

cmp ax,DECISION_ZONE 
jle first_isjbest 

sub bx,cx 

camp bx,DECISiON_ZONE 
jle second_isjDest 



third_isjbest: 



third_isjoJc: 



ntov cx,min_index_2 
cmp CX,MAX_GOOD_CORR 
jb third_is_ok 
jmp nq_good_corr 



mov ax,CORR_HISTORY SIZE 
sub ax,cx ~" 
jmp end_fms 



third_not_smallest: 

crap cx, dx 

ja fourth_is_smallest 
jz fourth_not_smallest 



/First - Third 



r Second - Third 



Fourth One Is Smallest 



f ourth_is_smallest : 

sub ax,dx 

cmp ax, DECISION ZONE 
jle first_is_best 

sub bx,dx 

cmp bx, DECISION ZONE 



; First - Fourth 



r Second - Fourth 



(D 
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mov minjndex_3,cx 

loop faliingJtoqp_3 
jmp £ incUbestjnin 

not_new_min_3: 

sub ax,dx 

cap ax,BANDGRF 

jae JHjidjDestmin 

loop felling_ioop_3 
• ****************** 



Decide which one is best 

findjDestjnin: 

mov ax,min_0 
raov bx,min_l 
mov cx,min_2 

cam? ax,bx 

jae first_notjsmallest 
camp ax,cx 

jae first_not_smallest 
cmp ax,dx 

jae first_not_smallest 



-irst_is_pk: 



First one is smallest, and always best 
^ixst_isjDest: 

mov cx,minindexj) 
cmp cx,MAXJ30GD CORR 
jb firstis_pk 
jmp nqjgood_corr 

mov ax,CORR_HISTOKT SIZE 
sub ax, cx 
jmp end_fms 

first _not smallest: 

cmp bx, cx 
, 3« aecondjiot smallest 

+++ ignore third £ fourth correlations 
cmp bx,dx 

jae second_notjsmallest 




305 "'^Lp^, /First - Second 

ax,DECXSXCK ZONE 



® 1990 Walt Dianay Imaginaering 



SUBSTITUTE SHEET , 



WO 93/22762 



PCT/US93/03667 



- 139 - 



jle second_isJbest 

; Third - Fourth 

cap cXyDKCTSICW ZONE 
jle third is best 



f ourthJLsJbeat : 

toov cx,min_iiidex_3 

cap CK^MkKJSOODjCOBR 
jb fourth_is_ok 
yap no_good_corr 

fourth_is_pk: 

mov ax, CORR_HISTORY SIZE 
sub ax,cx ~~ 
jnip end^fins 

f ourth_not_sraaliest : 

cmp ax r OFFFFH 
jz no_good_corr 
jnp f irst_isjDest 

nq_good_corr : 

mov ax, 0 

end_fms : 

pop di 
ret 

^^et^period endp 
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START 

FLOOR MAT SENSOR DETECTS GUEST AFTER 
^PREDETERMINED PERIOD OF NO GUEST PRESEN1 




CLEAR BUFFERS 
SET IDEAL TEMPO 
POINT TO BEGIN SCORE 
FILL CHANNEL "NOTE ON" AND 
NOTE OFF" BUFFERS FOR EACH CHANNE 



WAIT FLAG 
INDICATES THAT 
"IVG - 128" 
DIGITIZED FIRST 
FIELD 



IF SCORE 


EXHAUSTED 


AND MIB EMPTY 


THEN 


QUIT 



EXAMINE NOTE ON 
BUFFERS IF NOT FULL, 
FETCH FROM SCORE 
AND EXPAND BUFFERS 



ADVANCE VIDEO FRAME 
IF NECESSARY 

1 



N 




SEARCH FOR CHANGED PIXELS 
X STEP SIZE = 4 COLUMNS 
Y STEP SIZE = 6 ROWS 



COMPUTE SCALAR 
S =X L + Y L + INV.(Xr) + Yr 



STORE SCALAR IN 200 
POSITION CIRCULAR BUFFER 



CORRELATE 30 SCALARS OVER 
ALL 200 SCALARS IN BUFFER 



HAVE 2 BART 
ALREADY BEEN PLAYED ? 
(DOWNBEAT) 



COMPARE TEMPO TO PREVIOUS 
TEMPO SENT TO MIB 



IF TEMPO DIFFERENT, WRITE NEW 
TEMPO TO MIB 



ACCUMULATE NUMBER 
OF BEATS FROM MIB 

1 



{2> 



Fig.3 
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LOOP TWICE 



SAMPLE FIRST WINDOW 
BEGIN WITH VIDEO PIXEL ROW = 50 
VIDEO PIXEL COLUMN = 25 



SAMPLE SECOND WINDOW 
BEGIN WITH VIDEO PIXEL ROW = 50 
VIDEO PIXEL COLUMN = 208 



SEARCH FOR VALID PIXEL 

READ NEW PIXEL FROM "IVG-128" 
X STEP SIZE = 4 COLUMNS Y STEP SIZE = 6 ROWS ■ 
LOAD OLD PIXEL VALUE FROM PIXEL SAMPLE BUFFER 

WRITE NEW PIXEL FROM "IVG-128" INTO PIXEL SAMPLE BUFFER 
IS DIFFERENCE > 10 HEX ? 



NO VALID PIXEL FOUND 
WINDOW FINISHED - USE WINDOW 
CF N TROI D FR OM I AST FRAMF 



FOUND VALID PIXEL 



PERFORM SAMPLING 

SAMPLE 10 ROWS, USE STEP INCREMENTS 

IS DIFFERENCE > 10 HEX ? 
WRITE NEW PIXEL INTO PIXEL SAMPLE BUFFER 



IF DIFF. IS > 10 HEX, 
INCREMENT PIXEL COUNT 
ADD X INDEX TO X SUM 
ADD Y INDEX TO Y SUM 




CONTINUE STEPWISE LOADING OF "IVG-128" PIXELS 
WRITE OVER CORRESPONDING OLD PIXELS IN PIXEL SAMPLE BUFFER 

UNTIL FINISHED WITH WINDOW ' ; 



COMPUTE WINDOW CENTROID 
DIVIDE X INDEX BY PIXEL COUNT 
DIVIDE Y INDEX BY PIXEL COUNT 



REPEAT LOOP FOR SECOND WINDOW 



Fig.4 



HAVE AT LEAST 4 VALID 

PIXELS BEEN FOUND 

LEFT + RIGHT COUNTS > 3' 
? 

Y 



WRITE MIB WITH) 
MINIMUM TEMPO' 
& PROCEED TO 
ACCUMULATE 
iMLS I 



OVERRIDE STORED VOLUME (VELOCITY) FOR EACH CHANNEL 



CONTINUE MAIN PROGRAM LOOP WITH COMPUTATION OF SCALARSl 

3 
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m 

START 
MB 

8 CHANNELS WITH CORRESPONDING BUFFERS 
BUFFERS ALL EMPTY 




MIB INSTRUCTED FROM MAIN PROGRAM LOOP TO COMMENCE PLAYING 
GENERIC INTERRUPT TRIGGERED — 4 OF THE 8 BUFFERS LOADED 
WITH FIRST COMMAND AND COUNTDOWN TIMES FROM CPU BUFFERS 
(4 PAIRS OF "NOTE ON" AND "NOTE OFF' CIRCULAR BUFFERS) 



A COMMAND IS EXECUTED ( NOTE ON" OR "NOTE OFF') 



I 



MIB CHANNEL SPECIFIC INTERRUPT TRIGGERED 
CORRESPONDING TO EMPTY CHANNEL BUFFER 



INTERRUPT DIRECTS CPU TO "NOTE ON" AND "NOTE OFF" 
CIRCUUR BUFFER PAIR CORRESPONDING TO EMPTY CHANNEL 

T 

CPU CHECKS ABSOLUTE TIME OF NEXT COMMAND IN EACH OF 

THE "NOTE ON" AND "NOTE OFf CIRCULAR BUFFERS TO 
CHOOSE NEXT COMM AND 

1 

CPU COMPARES ABSOLUTE TIME OF CHOSEN COMMAND WITH 
ABSOLUTE TIME OF LAST COMMAND SENT TO MIB 




OVERRIDE VOLUME WITH STORED 
CHANNEL VOLUME (VELOCITY) 



SEND PROCESSED NOTE COMMAND TO MIB 



INCREMENT POINTER FOR CHOSEN CIRCULAR BUFFER | 



RESET INTERRUPT 
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SET INDEX "TO POINT TO 
MOST RECENT SCALAR 



SUBTRACT EACH OF 30 SCALARS BEGINNING 
WITH INDEX FROM 30 SCALARS BEGINNING 
WITH MOST RECENT SCALAR 



SUM THE SQUARES OF THE DIFFERENCES 



STORE THE SUM IN A 170 POSITION BUFFER 



Fig. 6 




COMPUTE AND ADD 
NEWEST SUM AND 
SUBTRACT OLDEST 



"DONE WITH ^\^N0 
\LL 170 SCALAF 



INCREMENT THE 
INDEX TO POINT TO 
NEXT SCALAR 



POINT TO T 
SUM IN 17( 
BUFFER (X N 


HE FIRST 
3 POSITIOh 
) 






FIND FIRST 


MAXIMUM h 


I 




FIND FIRST 


MINIMUM H 



[ 



{8> 



FIND SECOND MAXIMUM 

I 



FIND SECOND MINIMUM 

I 



CHOOSE SMALLEST MINIMUM 




DO NOT 
UPDATE 
TEMPO 



> 
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The invention permits the generation of 
multipurpose control signals by tracking movement 
that occurs within a field of view. In particular, 
a f 'Guest Controlled Orchestra' ? utilizing these 
inventive principles permits a layman guest to 
step into the shoes of an orchestra conductor, and 
through image processing, conduct the performance 
of a prerecorded music score. A video camera 
captures a field of view encompassing the guest 
for generation of a digital image. The field of 
view is sampled in left and right windows and the 
intensity of pixels within the windows are 
compared with a past image to determine if 
intensity change exceeds a predetermined 
threshold. A center of movement is computed for 
each window by averaging coordinates of each such 
pixel, and the centers of movement stored for 
future use. By analyzing change in centers of 
movement, tempo and volume are derived . Volume is 
derived from the quantity of pixels that 
correspond to the predetermined intensity change, 
and which therefore represent movement. 
Prerecorded audio data are formatted into MIDI 
audio commands, and together with video frame 
advance commands, are processed and output in 
response to these derived signals. 
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